Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 4.x #1432

Open
wants to merge 41 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
345ca8c
Convert shell plugin to internal plugin (#975)
Jul 22, 2022
54ab0c6
Remove old UI (#984)
Jul 22, 2022
376a820
Merge branch 'master' into 4.x
Dec 9, 2023
4dcd2c7
Shell command needs proper usage and desc to be valid
Dec 9, 2023
055f0e1
Release a light image tag (#988)
vcastellm Dec 12, 2023
7af72c3
Upgrade react admin to v4 (#1436)
vcastellm Feb 3, 2024
2f40b60
Use exponential backoff for retries (#1433)
vcastellm Feb 3, 2024
1c45fdf
Handle ip changes (#1446)
ivan-kripakov-m10 Feb 3, 2024
7ee3ab6
Remove shell bin building because it's builtin
vcastellm Feb 3, 2024
1377bcd
Fix windows build
vcastellm Feb 4, 2024
7a2d490
Clean up space in CI
vcastellm Feb 4, 2024
c35d31e
Move config init to agent command (#1465)
vcastellm Feb 6, 2024
665c865
Fix disabled (#1467)
vcastellm Feb 8, 2024
41c82f3
Docs
vcastellm Feb 8, 2024
586f72d
Bump module to v4 (#1470)
vcastellm Feb 10, 2024
f05aefe
Embed http plugin (#1471)
vcastellm Feb 10, 2024
03fb813
Merge branch 'main' into 4.x
vcastellm Feb 10, 2024
aec162a
Show all Job fields
vcastellm Feb 11, 2024
86ce37f
Refactor buttons for admin v4
vcastellm Feb 11, 2024
9a1bc04
Build UI
vcastellm Feb 11, 2024
fccc1fb
4.0 Banner and blog post
vcastellm Feb 11, 2024
f7eaff0
Build with docker node
vcastellm Feb 11, 2024
37fd38b
Docs v4 (#1473)
vcastellm Feb 11, 2024
203e31d
Reuse http clients with the same configuration (#1474)
vcastellm Feb 12, 2024
b71cf98
Update OpenAPI spec to v3.1.0 and add ACLs spec
vcastellm Feb 25, 2024
b7235ee
Merge branch '4.x' of github.com:distribworks/dkron into 4.x
vcastellm Feb 25, 2024
794a178
Refactor location of types as they not only belong to plugins, but us…
vcastellm Feb 25, 2024
a0717ae
Move member type to types
vcastellm Feb 25, 2024
6426308
Generate client
vcastellm Feb 25, 2024
4fcd8f7
protoc line
vcastellm Feb 25, 2024
aae95b9
Extend protobuf defs for ACLs
vcastellm Mar 24, 2024
b1ce3da
Token fields
vcastellm Mar 24, 2024
fbac360
Typo
vcastellm Mar 31, 2024
ede6770
Replace deprecated protobuf package
vcastellm Mar 31, 2024
fb49e86
Fix import
vcastellm Mar 31, 2024
e2a86ac
Token Type
vcastellm Mar 31, 2024
b2d5928
Use accessor for tokens
vcastellm Mar 31, 2024
d46947b
The policy should be just a ref
vcastellm Mar 31, 2024
a2face7
Consider existing Job
vcastellm Apr 1, 2024
5ffe0ec
UI fs as param
vcastellm Apr 1, 2024
8324be2
Minor improvement in RabbitMQ executor (#1500)
ivan-kripakov-m10 Jun 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 7 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ jobs:
TAG=${GITHUB_REF#refs/tags/}
echo ::set-output name=tag_name::${TAG}

- name: Clean up
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"

- name: Set up QEMU
uses: docker/setup-qemu-action@v1

Expand Down
60 changes: 42 additions & 18 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ release:

builds:
- &xbuild
main: ./builtin/bins/dkron-executor-http/
id: dkron-executor-http
binary: dkron-executor-http
main: .
id: dkron
binary: dkron
env:
- CGO_ENABLED=0
goos:
Expand All @@ -22,12 +22,7 @@ builds:
goarm:
- '7'
ldflags:
- -s -w

- <<: *xbuild
main: ./builtin/bins/dkron-executor-shell/
id: dkron-executor-shell
binary: dkron-executor-shell
- -s -w -X github.com/distribworks/dkron/v4/dkron.Version={{.Version}} -X github.com/distribworks/dkron/v4/dkron.Codename=Abaniko

- <<: *xbuild
main: ./builtin/bins/dkron-executor-rabbitmq/
Expand Down Expand Up @@ -74,13 +69,6 @@ builds:
id: dkron-processor-fluent
binary: dkron-processor-fluent

- <<: *xbuild
main: .
id: dkron
binary: dkron
ldflags:
- -s -w -X github.com/distribworks/dkron/v3/dkron.Version={{.Version}} -X github.com/distribworks/dkron/v3/dkron.Codename=Abaniko

nfpms:
-
vendor: Distributed Works
Expand Down Expand Up @@ -126,8 +114,6 @@ dockers:
goarch: amd64
ids: &docker-ids
- dkron
- dkron-executor-http
- dkron-executor-shell
- dkron-executor-rabbitmq
- dkron-executor-nats
- dkron-executor-kafka
Expand Down Expand Up @@ -161,6 +147,38 @@ dockers:
build_flag_templates:
- --platform=linux/arm/v7

- image_templates:
- dkron/{{ .ProjectName }}:{{ .Version }}-light-amd64
dockerfile: Dockerfile.release
use: buildx
goos: linux
goarch: amd64
ids: &docker-ids
- dkron
build_flag_templates:
- --platform=linux/amd64

- image_templates:
- dkron/{{ .ProjectName }}:{{ .Version }}-light-arm64
dockerfile: Dockerfile.release
use: buildx
goos: linux
goarch: arm64
ids: *docker-ids
build_flag_templates:
- --platform=linux/arm64/v8

- image_templates:
- dkron/{{ .ProjectName }}:{{ .Version }}-light-armv7
dockerfile: Dockerfile.release
use: buildx
goos: linux
goarch: arm
goarm: '7'
ids: *docker-ids
build_flag_templates:
- --platform=linux/arm/v7

docker_manifests:
- name_template: dkron/{{ .ProjectName }}:{{ .Version }}
image_templates:
Expand All @@ -174,6 +192,12 @@ docker_manifests:
- dkron/{{ .ProjectName }}:{{ .Version }}-arm64
- dkron/{{ .ProjectName }}:{{ .Version }}-armv7

- name_template: dkron/{{ .ProjectName }}:{{ .Version }}-light
image_templates:
- dkron/{{ .ProjectName }}:{{ .Version }}-light-amd64
- dkron/{{ .ProjectName }}:{{ .Version }}-light-arm64
- dkron/{{ .ProjectName }}:{{ .Version }}-light-armv7

changelog:
sort: asc
filters:
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM golang:1.18
LABEL maintainer="Victor Castell <victor@victorcastell.com>"
FROM golang:1.21
LABEL maintainer="Victor Castell <0x@vcastellm.xyz>"

EXPOSE 8080 8946

Expand Down
15 changes: 15 additions & 0 deletions Dockerfile.light.hub
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM alpine:3.19
LABEL maintainer="Victor Castell <0x@vcastellm.xyz>"

RUN set -x \
&& buildDeps='bash ca-certificates openssl tzdata' \
&& apk add --no-cache --update $buildDeps

EXPOSE 8080 8946

ENV SHELL /bin/bash
COPY dkron /usr/local/bin/

ENTRYPOINT ["/usr/local/bin/dkron"]

CMD ["--help"]
4 changes: 2 additions & 2 deletions Dockerfile.release
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM alpine:3.15
LABEL maintainer="Victor Castell <victor@victorcastell.com>"
FROM alpine:3.19
LABEL maintainer="Victor Castell <0x@vcastellm.xyz>"

RUN set -x \
&& buildDeps='bash ca-certificates openssl tzdata' \
Expand Down
32 changes: 21 additions & 11 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,14 @@ clean:
GOBIN=`pwd` go clean -i ./builtin/...
GOBIN=`pwd` go clean

.PHONY: doc apidoc test ui updatetestcert
doc:
#scripts/run doc --dir website/docs/cli
cd website; yarn build --out-dir ../public
.PHONY: docs apidoc test ui updatetestcert
docs:
# scripts/run doc --dir website/docs/cli

# Build with docker while bun reach compatibility with docusaurs
# cd website; bun run yarn build --out-dir ../public
docker run -v `pwd`:/app --workdir /app/website node yarn build --out-dir ../public

ghp-import -p public

test:
Expand All @@ -72,24 +76,30 @@ localtest:

updatetestcert:
wget https://badssl.com/certs/badssl.com-client.p12 -q -O badssl.com-client.p12
openssl pkcs12 -in badssl.com-client.p12 -nocerts -nodes -passin pass:badssl.com -out builtin/bins/dkron-executor-http/testdata/badssl.com-client-key-decrypted.pem
openssl pkcs12 -in badssl.com-client.p12 -nokeys -passin pass:badssl.com -out builtin/bins/dkron-executor-http/testdata/badssl.com-client.pem
openssl pkcs12 -in badssl.com-client.p12 -nocerts -nodes -passin pass:badssl.com -out plugin/http/testdata/badssl.com-client-key-decrypted.pem
openssl pkcs12 -in badssl.com-client.p12 -nokeys -passin pass:badssl.com -out plugin/http/testdata/badssl.com-client.pem
rm badssl.com-client.p12

ui/node_modules: ui/package.json
cd ui; npm install
cd ui; bun install
# touch the directory so Make understands it is up to date
touch ui/node_modules

dkron/ui-dist: ui/node_modules ui/public/* ui/src/* ui/src/*/*
cd ui; npm run-script build
rm -rf dkron/ui-dist
cd ui; bun run build --out-dir ../dkron/ui-dist

plugin/types/%.pb.go: proto/%.proto
protoc -I proto/ --go_out=plugin/types --go_opt=paths=source_relative --go-grpc_out=plugin/types --go-grpc_opt=paths=source_relative $<
proto: types/dkron.pb.go types/executor.pb.go types/pro.pb.go

types/%.pb.go: proto/%.proto
protoc -I proto/ --go_out=types --go_opt=paths=source_relative --go-grpc_out=types --go-grpc_opt=paths=source_relative $<

client:
oapi-codegen -package client website/static/openapi/openapi.yaml > client/client.go

ui: dkron/ui-dist

main: dkron/ui-dist plugin/types/dkron.pb.go plugin/types/executor.pb.go *.go */*.go */*/*.go */*/*/*.go
main: dkron/ui-dist types/dkron.pb.go types/executor.pb.go *.go */*.go */*/*.go */*/*/*.go
GOBIN=`pwd` go install ./builtin/...
go mod tidy
go build main.go
19 changes: 10 additions & 9 deletions builtin/bins/dkron-executor-gcppubsub/gcppubsub.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ import (
"fmt"

"cloud.google.com/go/pubsub"
dkplugin "github.com/distribworks/dkron/v3/plugin"
dktypes "github.com/distribworks/dkron/v3/plugin/types"
dkplugin "github.com/distribworks/dkron/v4/plugin"
dktypes "github.com/distribworks/dkron/v4/types"
)

// GCPPubSub plugin publish message to topic when Execute method is called.
Expand All @@ -24,12 +24,13 @@ const (

// Execute Process method of the plugin
// "executor": "gcppubsub",
// "executor_config": {
// "project": "project-id",
// "topic": "topic-name",
// "data": "aGVsbG8gd29ybGQ=" // Optional. base64 encoded data
// "attributes": "{\"hello\":\"world\",\"waka\":\"paka\"}" // JSON serialized attributes
// }
//
// "executor_config": {
// "project": "project-id",
// "topic": "topic-name",
// "data": "aGVsbG8gd29ybGQ=" // Optional. base64 encoded data
// "attributes": "{\"hello\":\"world\",\"waka\":\"paka\"}" // JSON serialized attributes
// }
func (g *GCPPubSub) Execute(args *dktypes.ExecuteRequest, _ dkplugin.StatusHelper) (*dktypes.ExecuteResponse, error) {
out, err := g.ExecuteImpl(args)
resp := &dktypes.ExecuteResponse{Output: out}
Expand Down Expand Up @@ -91,7 +92,7 @@ func configToPubSubMessage(config map[string]string) (*pubsub.Message, error) {

var attributes map[string]string
if attributesJSON != "" {
if err := json.Unmarshal([]byte(attributesJSON), &attributes); err != nil {
if err := json.Unmarshal([]byte(attributesJSON), &attributes); err != nil {
return nil, fmt.Errorf("invalid attributes JSON: %w", err)
}
msg.Attributes = attributes
Expand Down
2 changes: 1 addition & 1 deletion builtin/bins/dkron-executor-gcppubsub/gcppubsub_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"testing"

"cloud.google.com/go/pubsub"
dktypes "github.com/distribworks/dkron/v3/plugin/types"
dktypes "github.com/distribworks/dkron/v4/types"
"github.com/stretchr/testify/assert"
)

Expand Down
2 changes: 1 addition & 1 deletion builtin/bins/dkron-executor-gcppubsub/main.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package main

import (
dkplugin "github.com/distribworks/dkron/v3/plugin"
dkplugin "github.com/distribworks/dkron/v4/plugin"
"github.com/hashicorp/go-plugin"
)

Expand Down
17 changes: 9 additions & 8 deletions builtin/bins/dkron-executor-grpc/grpc.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
"time"

"github.com/armon/circbuf"
dkplugin "github.com/distribworks/dkron/v3/plugin"
dktypes "github.com/distribworks/dkron/v3/plugin/types"
dkplugin "github.com/distribworks/dkron/v4/plugin"
dktypes "github.com/distribworks/dkron/v4/types"
"github.com/fullstorydev/grpcurl"
"github.com/jhump/protoreflect/grpcreflect"
"github.com/pkg/errors"
Expand All @@ -33,12 +33,13 @@ type GRPC struct{}

// Execute Process method of the plugin
// "executor": "grpc",
// "executor_config": {
// "url": "127.0.0.1:9000/demo.DemoService/Demo", // Request url
// "body": "", // POST body
// "timeout": "30", // Request timeout, unit seconds
// "expectCode": "0", // Expect response code, any of the described here https://grpc.github.io/grpc/core/md_doc_statuscodes.html
// }
//
// "executor_config": {
// "url": "127.0.0.1:9000/demo.DemoService/Demo", // Request url
// "body": "", // POST body
// "timeout": "30", // Request timeout, unit seconds
// "expectCode": "0", // Expect response code, any of the described here https://grpc.github.io/grpc/core/md_doc_statuscodes.html
// }
func (g *GRPC) Execute(args *dktypes.ExecuteRequest, _ dkplugin.StatusHelper) (*dktypes.ExecuteResponse, error) {
out, err := g.ExecuteImpl(args)
resp := &dktypes.ExecuteResponse{Output: out}
Expand Down
9 changes: 5 additions & 4 deletions builtin/bins/dkron-executor-grpc/grpc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@ package main

import (
"context"
"github.com/distribworks/dkron/v3/builtin/bins/dkron-executor-grpc/test"
dktypes "github.com/distribworks/dkron/v3/plugin/types"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
"net"
"testing"

"github.com/distribworks/dkron/v4/builtin/bins/dkron-executor-grpc/test"
dktypes "github.com/distribworks/dkron/v4/types"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
)

type DemoServer struct {
Expand Down
2 changes: 1 addition & 1 deletion builtin/bins/dkron-executor-grpc/main.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package main

import (
dkplugin "github.com/distribworks/dkron/v3/plugin"
dkplugin "github.com/distribworks/dkron/v4/plugin"
"github.com/hashicorp/go-plugin"
)

Expand Down
18 changes: 0 additions & 18 deletions builtin/bins/dkron-executor-http/main.go

This file was deleted.

17 changes: 9 additions & 8 deletions builtin/bins/dkron-executor-kafka/kafka.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
"github.com/Shopify/sarama"
"github.com/armon/circbuf"

dkplugin "github.com/distribworks/dkron/v3/plugin"
dktypes "github.com/distribworks/dkron/v3/plugin/types"
dkplugin "github.com/distribworks/dkron/v4/plugin"
dktypes "github.com/distribworks/dkron/v4/types"
)

const (
Expand All @@ -26,12 +26,13 @@ type Kafka struct {

// Execute Process method of the plugin
// "executor": "kafka",
// "executor_config": {
// "brokerAddress": "192.168.59.103:9092", // kafka broker url
// "key": "",
// "message": "",
// "topic": "publishTopic"
// }
//
// "executor_config": {
// "brokerAddress": "192.168.59.103:9092", // kafka broker url
// "key": "",
// "message": "",
// "topic": "publishTopic"
// }
func (s *Kafka) Execute(args *dktypes.ExecuteRequest, cb dkplugin.StatusHelper) (*dktypes.ExecuteResponse, error) {

out, err := s.ExecuteImpl(args)
Expand Down
4 changes: 2 additions & 2 deletions builtin/bins/dkron-executor-kafka/kafka_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"fmt"
"testing"

dktypes "github.com/distribworks/dkron/v3/plugin/types"
dktypes "github.com/distribworks/dkron/v4/types"
)

func TestProduceExecuteWithKey(t *testing.T) {
Expand Down Expand Up @@ -44,4 +44,4 @@ func TestProduceExecuteWithoutKey(t *testing.T) {
if err != nil {
t.Fatal(err)
}
}
}
2 changes: 1 addition & 1 deletion builtin/bins/dkron-executor-kafka/main.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package main

import (
dkplugin "github.com/distribworks/dkron/v3/plugin"
dkplugin "github.com/distribworks/dkron/v4/plugin"
"github.com/hashicorp/go-plugin"
)

Expand Down