Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Rust Wasm Plugin Build & Publish Action #1483

Merged
merged 4 commits into from
Nov 7, 2024
Merged

Implement Rust Wasm Plugin Build & Publish Action #1483

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fb20c35
build
007gzs Nov 6, 2024
16fd090
Merge branch 'alibaba:main' into rust_build_publish
007gzs Nov 6, 2024
4406c98
comment
007gzs Nov 6, 2024
ada695c
Merge branch 'main' into rust_build_publish
007gzs Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 39 additions & 7 deletions .github/workflows/build-and-push-wasm-plugin-image.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,16 @@ name: Build and Push Wasm Plugin Image
on:
push:
tags:
- "wasm-go-*-v*.*.*" # 匹配 wasm-go-{pluginName}-vX.Y.Z 格式的标签
- "wasm-*-*-v*.*.*" # 匹配 wasm-{go|rust}-{pluginName}-vX.Y.Z 格式的标签
workflow_dispatch:
inputs:
plugin_type:
description: 'Type of the plugin'
required: true
type: choice
options:
- go
- rust
plugin_name:
description: 'Name of the plugin'
required: true
Expand All @@ -23,32 +30,40 @@ jobs:
env:
IMAGE_REGISTRY_SERVICE: ${{ vars.IMAGE_REGISTRY || 'higress-registry.cn-hangzhou.cr.aliyuncs.com' }}
IMAGE_REPOSITORY: ${{ vars.PLUGIN_IMAGE_REPOSITORY || 'plugins' }}
RUST_VERSION: 1.82
GO_VERSION: 1.19
TINYGO_VERSION: 0.28.1
ORAS_VERSION: 1.0.0
steps:
- name: Set plugin_name and version from inputs or ref_name
- name: Set plugin_type、 plugin_name and version from inputs or ref_name
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use full width character in english comments

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

id: set_vars
run: |
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
plugin_type="${{ github.event.inputs.plugin_type }}"
plugin_name="${{ github.event.inputs.plugin_name }}"
version="${{ github.event.inputs.version }}"
builder_image="higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/wasm-rust-builder:rust${{ env.RUST_VERSION }}-oras${{ env.ORAS_VERSION }}"
else
ref_name=${{ github.ref_name }}
plugin_type=${ref_name#*-} # 删除插件类型前面的字段(wasm-)
plugin_type=${plugin_type%-*} # 删除插件类型后面的字段(-{plugin_name}-vX.Y.Z)
plugin_name=${ref_name#*-*-} # 删除插件名前面的字段(wasm-go-)
plugin_name=${plugin_name%-*} # 删除插件名后面的字段(-vX.Y.Z)
version=$(echo "$ref_name" | awk -F'v' '{print $2}')
builder_image="higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/wasm-go-builder:go${{ env.GO_VERSION }}-tinygo${{ env.TINYGO_VERSION }}-oras${{ env.ORAS_VERSION }}"
fi

echo "PLUGIN_TYPE=$plugin_type" >> $GITHUB_ENV
echo "PLUGIN_NAME=$plugin_name" >> $GITHUB_ENV
echo "VERSION=$version" >> $GITHUB_ENV
echo "BUILDER_IMAGE=$builder_image" >> $GITHUB_ENV

- name: Checkout code
uses: actions/checkout@v3

- name: File Check
run: |
workspace=${{ github.workspace }}/plugins/wasm-go/extensions/${PLUGIN_NAME}
workspace=${{ github.workspace }}/plugins/wasm-${PLUGIN_TYPE}/extensions/${PLUGIN_NAME}
push_command="./plugin.tar.gz:application/vnd.oci.image.layer.v1.tar+gzip"

# 查找spec.yaml
Expand All @@ -75,10 +90,10 @@ jobs:

echo "PUSH_COMMAND=\"$push_command\"" >> $GITHUB_ENV

- name: Run a wasm-go-builder
- name: Run a wasm-builder
env:
PLUGIN_NAME: ${{ env.PLUGIN_NAME }}
BUILDER_IMAGE: higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/wasm-go-builder:go${{ env.GO_VERSION }}-tinygo${{ env.TINYGO_VERSION }}-oras${{ env.ORAS_VERSION }}
BUILDER_IMAGE: ${{ env.BUILDER_IMAGE }}
run: |
docker run -itd --name builder -v ${{ github.workspace }}:/workspace -e PLUGIN_NAME=${{ env.PLUGIN_NAME }} --rm ${{ env.BUILDER_IMAGE }} /bin/bash

Expand All @@ -93,15 +108,15 @@ jobs:
echo "TargetImage=${target_image}"
echo "TargetImageLatest=${target_image_latest}"

cd ${{ github.workspace }}/plugins/wasm-go/extensions/${PLUGIN_NAME}
cd ${{ github.workspace }}/plugins/wasm-${PLUGIN_TYPE}/extensions/${PLUGIN_NAME}
if [ -f ./.buildrc ]; then
echo 'Found .buildrc file, sourcing it...'
. ./.buildrc
else
echo '.buildrc file not found'
fi
echo "EXTRA_TAGS=${EXTRA_TAGS}"

if [ "${PLUGIN_TYPE}" == "go" ]; then
command="
set -e
cd /workspace/plugins/wasm-go/extensions/${PLUGIN_NAME}
Expand All @@ -112,4 +127,21 @@ jobs:
oras push ${target_image} ${push_command}
oras push ${target_image_latest} ${push_command}
"
elif [ "${PLUGIN_TYPE}" == "rust" ]; then
command="
set -e
cd /workspace/plugins/wasm-rust/extensions/${PLUGIN_NAME}
cargo build --target wasm32-wasi --release
cp target/wasm32-wasi/release/*.wasm plugin.wasm
tar czvf plugin.tar.gz plugin.wasm
echo ${{ secrets.REGISTRY_PASSWORD }} | oras login -u ${{ secrets.REGISTRY_USERNAME }} --password-stdin ${{ env.IMAGE_REGISTRY_SERVICE }}
oras push ${target_image} ${push_command}
oras push ${target_image_latest} ${push_command}
"
else

command="
echo "unkown type ${PLUGIN_TYPE}"
"
fi
docker exec builder bash -c "$command"
38 changes: 38 additions & 0 deletions plugins/wasm-rust/DockerfileBuilder
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need another Dockerfile to build?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I referred to the builder used for packaging and publishing in the go-lang https://github.com/alibaba/higress/blob/main/plugins/wasm-go/DockerfileBuilder

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dockerfile is enough, no need additional files

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both Dockerfile and DockerfileBuilder are for building an artifact named plugin.wasm, but the oras is a client which aims to publish artifact to registry. It means no matter which language is, the only different is how to build an artifact but using same way to publish artifact.

In a word, I think the oras should be a independent image, but not mixed in builders.

cc @johnlanni

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

builder mainly facilitates developers by eliminating the need to download corresponding versions of oras, as well as Rust build tools, etc

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For developers, devcontainer might be the best solution and I still think it should keep simple for OCI building

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you're right, developers just need the Dockerfile, but it seems that the DockerfileBuilder used for building images allows the workflow for publishing wasm plugins to remain consistent with Go. From this perspective, it still makes sense

ARG RUST_VERSION
ARG ORAS_VERSION
ARG HIGRESS_VERSION

ARG BASE_IMAGE=rust:${RUST_VERSION:-1.82}
FROM $BASE_IMAGE

LABEL rust_version=$RUST_VERSION oras_version=$ORAS_VERSION

RUN apt-get update \
&& apt-get install -y wget gcc gcc-multilib llvm clang \
&& rustup target add wasm32-wasi \
&& rm -rf /var/lib/apt/lists/*

RUN arch="$(dpkg --print-architecture)"; arch="${arch##*-}"; \
rust_version=${RUST_VERSION:-1.82}; \
oras_version=${ORAS_VERSION:-1.0.0}; \
higress_version=${HIGRESS_VERSION:-1.0.0-rc}; \
echo "arch: '$arch'"; \
echo "rust rust_version: '$rust_version'"; \
echo "oras_version: '$oras_version'"; \
echo "higress_version: '$higress_version'"; \
case "$arch" in \
'amd64') \
oras_url="https://github.com/oras-project/oras/releases/download/v$oras_version/oras_${oras_version}_linux_amd64.tar.gz"; \
;; \
'arm64') \
oras_url="https://github.com/oras-project/oras/releases/download/v$oras_version/oras_${oras_version}_linux_arm64.tar.gz"; \
;; \
*) echo >&2 "error: unsupported architecture '$arch' "; exit 1 ;; \
esac; \
echo "oras_url: '$oras_url'"; \
wget -O oras.tgz "$oras_url" --progress=dot:giga; \
tar -C /usr/local/bin -xzf oras.tgz && rm -rf oras.tgz; \
echo "done";

ENV PATH=$PATH:/usr/local/bin
18 changes: 18 additions & 0 deletions plugins/wasm-rust/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
PLUGIN_NAME ?= say-hello
BUILDER_REGISTRY ?= higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/
REGISTRY ?= higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/
RUST_VERSION ?= 1.82
ORAS_VERSION ?= 1.0.0
HIGRESS_VERSION ?= 1.0.0-rc
BUILDER ?= ${BUILDER_REGISTRY}wasm-rust-builder:rust${RUST_VERSION}-oras${ORAS_VERSION}
BUILD_TIME := $(shell date "+%Y%m%d-%H%M%S")
COMMIT_ID := $(shell git rev-parse --short HEAD 2>/dev/null)
IMAGE_TAG = $(if $(strip $(PLUGIN_VERSION)),${PLUGIN_VERSION},${BUILD_TIME}-${COMMIT_ID})
Expand All @@ -21,3 +26,16 @@ lint-base:
lint:
cargo fmt --all --check --manifest-path extensions/${PLUGIN_NAME}/Cargo.toml
cargo clippy --workspace --all-features --all-targets --manifest-path extensions/${PLUGIN_NAME}/Cargo.toml

builder:
docker buildx build --no-cache \
--platform linux/amd64,linux/arm64 \
--build-arg RUST_VERSION=$(RUST_VERSION) \
--build-arg ORAS_VERSION=$(ORAS_VERSION) \
--build-arg HIGRESS_VERSION=$(HIGRESS_VERSION) \
-f DockerfileBuilder \
-t ${BUILDER} \
--push \
.
@echo ""
@echo "image: ${BUILDER}"
Loading