-
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(transformer): transformer-based backend #1015
Labels
Comments
mudler
added a commit
that referenced
this issue
Sep 9, 2023
Related to: #1015 Signed-off-by: Ettore Di Giacinto <[email protected]>
mudler
added a commit
that referenced
this issue
Sep 9, 2023
Related to: #1015 Signed-off-by: Ettore Di Giacinto <[email protected]>
mudler
added a commit
that referenced
this issue
Sep 10, 2023
Is Vllm fully integrated? vLLM begins to seem to be many AWQ quantized model's first choice recently |
yep, going into the next release soon! |
mudler
added a commit
that referenced
this issue
Sep 22, 2023
This particularly useful to set AWQ **Description** Follow up of #1015 **Notes for Reviewers** **[Signed commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)** - [ ] Yes, I signed my commits. <!-- Thank you for contributing to LocalAI! Contributing Conventions: 1. Include descriptive PR titles with [<component-name>] prepended. 2. Build and test your changes before submitting a PR. 3. Sign your commits By following the community's contribution conventions upfront, the review process will be accelerated and your PR merged more quickly. --> --------- Signed-off-by: Ettore Di Giacinto <[email protected]>
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
It makes sense to have a backend which is tied to https://github.com/huggingface/transformers. This would increase the range of support for LLM that LocalAI already offers
Describe the solution you'd like
Ideally would be preferred to use a wrapper around transformers and xformers to avoid duplicating efforts.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: