Add multimodality support #2001
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
RTM |
DipFlip
pushed a commit
to DipFlip/anything-llm
that referenced
this pull request
Aug 4, 2024
* Add multimodality support * Add Bedrock, KoboldCpp,LocalAI,and TextWebGenUI multi-modal * temp dev build * patch bad import * noscrolls for windows dnd * noscrolls for windows dnd * update README * update README * add multimodal check
TuanBC
pushed a commit
to TuanBC/anything-llm
that referenced
this pull request
Aug 26, 2024
* Add multimodality support * Add Bedrock, KoboldCpp,LocalAI,and TextWebGenUI multi-modal * temp dev build * patch bad import * noscrolls for windows dnd * noscrolls for windows dnd * update README * update README * add multimodal check
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Type
Relevant Issues
resolves #558
What is in this change?
Supported models must be all-in-one models. If demand is high enough we will break out vision from LLM
Supported
Unsupported
Additional Information
The use of multi-modality in a non-multi-modal LLM will simply produce an error message in the UI from the provider.
The
DNDFileUploaderwas refactored to movefilesto a high scope as well as provide it as a context around the chatAPIsupport on the backend is not in this PR and will be done later once we know this to be stable.Local development docker testing
Supporting documentation on docs site (PR: 74
Implementation testing
Developer Validations
yarn lintfrom the root of the repo & committed changes