Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Improving the heterogenous-GPU feature doc #634

Merged
merged 3 commits into from
Feb 6, 2025

Conversation

nwangfw
Copy link
Collaborator

@nwangfw nwangfw commented Feb 6, 2025

Pull Request Description

  1. Replaced the [model_name] with an example deepseek-coder-7b in the heterogenous-gpu introduction doc.
  2. Added more explanation for the new label model.aibrix.ai/min_replicas, how to configure it, and whyminReplia defined in podAutoscaler doesn't work in heterogenous story.
  3. Corrected and added model.aibrix.ai/min_replicas in sample/heterogenous yaml
  4. Found a format issue, ..note:: command doesn't work. Removed it from doc.

Related Issues

Resolves: #[Insert issue number(s)]

Important: Before submitting, please complete the description above and review the checklist below.


Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

  • [Bug]: Corrections to existing functionality
  • [CI]: Changes to build process or CI pipeline
  • [Docs]: Updates or additions to documentation
  • [API]: Modifications to aibrix's API or interface
  • [CLI]: Changes or additions to the Command Line Interface
  • [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

  • PR title includes appropriate prefix(es)
  • Changes are clearly explained in the PR description
  • New and existing tests pass successfully
  • Code adheres to project style and best practices
  • Documentation updated to reflect changes (if applicable)
  • Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

@nwangfw nwangfw changed the title Ning/fix heter gpu doc [Docs] Improving the heterogenous-GPU feature doc Feb 6, 2025
Copy link
Collaborator

@zhangjyr zhangjyr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do need the label "model.aibrix.ai/min_replicas" to make heterogenous GPU working. Consider a configuration of (0xA10, 5xL20), and minimum configuration (1xA10, 0xL20). If we set minReplicas of A10 deployment to 1, the (0xA10, 5xL20) configuration can't be applied.

@nwangfw nwangfw force-pushed the ning/fix-heter-gpu-doc branch 3 times, most recently from 5cc4126 to a2ce701 Compare February 6, 2025 03:35
@nwangfw
Copy link
Collaborator Author

nwangfw commented Feb 6, 2025

We do need the label "model.aibrix.ai/min_replicas" to make heterogenous GPU working. Consider a configuration of (0xA10, 5xL20), and minimum configuration (1xA10, 0xL20). If we set minReplicas of A10 deployment to 1, the (0xA10, 5xL20) configuration can't be applied.

Great example! I have added the old config back and also added some more explanation. Please help to review it @zhangjyr

@nwangfw nwangfw force-pushed the ning/fix-heter-gpu-doc branch from a2ce701 to 3d4d9ed Compare February 6, 2025 03:46
@nwangfw nwangfw force-pushed the ning/fix-heter-gpu-doc branch from 3d4d9ed to e5e4ac8 Compare February 6, 2025 03:47
Copy link
Collaborator

@Jeffwan Jeffwan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@Jeffwan Jeffwan merged commit 2f39941 into main Feb 6, 2025
2 checks passed
@Jeffwan Jeffwan deleted the ning/fix-heter-gpu-doc branch February 6, 2025 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants