Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support auto-generating function pipelines for different workflow orchestrators #226

Closed
frankfarzan opened this issue Feb 5, 2020 · 4 comments
Labels
area/hydrate enhancement New feature or request triaged Issue has been triaged by adding an `area/` label

Comments

@frankfarzan
Copy link
Contributor

frankfarzan commented Feb 5, 2020

e.g.:

kpt fn export [cloud-build,jenkins-pipeline,github-actions]

This would generate e.g. Cloud Build configuration for running a pipeline of kpt functions.

There are various design options we should explore:

  1. Each workflow step runs the function wrapped with a utility that wires functions input/output in a shared volume
  2. Each workflow step runs the function directly using unique input/output files in a shared volume. As an example see:
    https://github.com/GoogleCloudPlatform/csp-config-management/blob/1.0.0/ci-pipeline-unstructured/cloudbuild.yaml
  3. A single workflow step that runs the entire kpt functions pipeline. This would lose some UX benefit of separate reporting/notifications in the workflow UI, but maybe acceptable if the workflow orchestrator is an implementation detail. For example, if everything is bubbled up into the GitHub UI instead of user going to Jenkins or Cloud Build. A technical challenge: Running docker-in-docker is not supported by all orchestrators.

A general issue is that workflow orchestrators do not typically support stdin/stdout, and the configuration function spec doesn't enforce a consistent flag name for input/output.

@frankfarzan frankfarzan added the enhancement New feature or request label Feb 5, 2020
@frankfarzan
Copy link
Contributor Author

  1. Mandate flags and/or env vars to functions spec:

-i, --input
-o, --output

Downside is that existing tools (e.g. kubectl, helm, kustomize) need to be wrapped.

@Liujingfang1
Copy link
Contributor

Liujingfang1 commented May 19, 2020

/assign @ZhuGongpu

@frankfarzan frankfarzan added the triaged Issue has been triaged by adding an `area/` label label May 27, 2020
@sankit2020 sankit2020 removed this from the Milestone 3 (Starts 2020-Q3) milestone Jun 24, 2020
@prachirp
Copy link
Contributor

@Liujingfang1 is this done?

@Liujingfang1
Copy link
Contributor

@prachirp Yes, this is done. The document is available https://googlecontainertools.github.io/kpt/guides/consumer/function/export/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/hydrate enhancement New feature or request triaged Issue has been triaged by adding an `area/` label
Projects
None yet
Development

No branches or pull requests

5 participants