Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add super pipeline sample #68

Merged
merged 15 commits into from
May 8, 2024
Merged

Add super pipeline sample #68

merged 15 commits into from
May 8, 2024

Conversation

revit13
Copy link
Collaborator

@revit13 revit13 commented May 5, 2024

Why are these changes needed?

This PR adds super pipeline sample including a new kfp component.

@revit13 revit13 marked this pull request as draft May 5, 2024 05:40
@revit13 revit13 marked this pull request as ready for review May 5, 2024 08:18
Copy link
Collaborator

@blublinsky blublinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


**Note** An example super pipeline that combines several transforms, `doc_id`, `ededup`, and `fdedup`, can be found in [superworkflow_dedups_sample_wf.py](../transform_workflows/superworkflows/superworkflow_dedups_sample_wf.py).

![super pipeline](super_pipeline.png)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the page is missing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Thanks

# Define the root of the local git clone for the common rules to be able
# know where they are running from.
REPOROOT=../../..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add include ../../.make.transforms_workflows ?

revit13 and others added 14 commits May 7, 2024 14:47
Signed-off-by: Revital Sur <[email protected]>
Co-authored-by: Mohammad Nassar <[email protected]>
Co-authored-by: Alexey Roytman <[email protected]>
Co-authored-by: Boris Lublinsky <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Signed-off-by: Revital Sur <[email protected]>
Copy link
Member

@roytman roytman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@daw3rd daw3rd merged commit c9466c1 into IBM:dev May 8, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants