Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Ingestion Load #47932

Open
3 tasks done
gnehil opened this issue Feb 14, 2025 · 0 comments · Fixed by #45937
Open
3 tasks done

[Feature] Ingestion Load #47932

gnehil opened this issue Feb 14, 2025 · 0 comments · Fixed by #45937
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@gnehil
Copy link
Contributor

gnehil commented Feb 14, 2025 •
edited
Loading

Search before asking

  • I had searched in the issues and found no similar issues.

Description

Load Job for new spark load.

Ingestion Load is used to load pre-processed data into doris.

Preprocessing refers to writing the result data to an external storage system after the data is processed according to the partitioning, bucketing and aggregation methods defined by the doris table.

The preprocessing is completed by the external system, and then the BE reads the data and converts it into segment files and saves it.

The basic flow is as follows:

Image

Use case

No response

Related issues

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[feature](load) new insgestion load gnehil/doris
1 participant
@gnehil