Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: experimental integration with helm-x #673

Merged
merged 1 commit into from
Jun 12, 2019
Merged

feat: experimental integration with helm-x #673

merged 1 commit into from
Jun 12, 2019

Conversation

mumoshu
Copy link
Collaborator

@mumoshu mumoshu commented Jun 11, 2019

This enhances helmfile so that it can:

  • Treat K8s manifests directories and Kustomize projects as charts
  • Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (feat: adhoc chart dependencies #649)
  • Add adhoc patches(JSON Patch or Strategic Merge Patch supported) to be applied to the K8s resources before sync/diff/template, without forking or modifyin chart(s) (feat: patching chart without forking it #650)

The usage is as outlined in https://github.com/mumoshu/helm-x/tree/master/examples/helmfile.

Add any or all of dependencies:, jsonPatches: and strategicMergePatches: so that it adds additional flags to helm calls that is only supported by helm x.

releases:
- name: kustomize
  chart: ../kustomize
- name: manifests
  chart: ../manifests
- name: foo
  chart: incubator/raw
  dependencies:
  - alias: bar
    chart: incubator/raw
  values:
  - values.yaml
  - bar:
      enabled: true
      resources:
      - apiVersion: v1
        kind: Pod
        metadata:
          name: bar
        spec:
          containers:
          - command:
            - sleep
            - 1000
            image: alpine:3.9.4
            imagePullPolicy: IfNotPresent
            name: bar
  jsonPatches:
  - target:
      version: v1
      kind: Pod
      name: foo
    patch:
    - op: replace
      path: /spec/containers/0/command
      value:
      - sleep
      - "123"
  strategicMergePatches:
  - apiVersion: v1
    kind: Pod
    metadata:
      name: bar
    spec:
      containers:
      - name: bar
        command:
        - sleep
        - "234"

You can alternatively provide source: path/to/patch.yaml for jsonPatches and strategicMergePatches items to externalize. Add a .gotmpl suffix like you would do for values files to enable templating.

To make this feature work, you must point --helm-binary to the helm-x binary like below:

$ helmfile --helm-binary ~/.helm/plugins/helm-x/bin/helm-x --log-level debug apply

The easiest way to grab the helm-x binary is to install it as a helm plugin:

$ helm plugin install https://github.com/mumoshu/helm-x

See helm-x for more details on it.

The integration should ideally be automatic. That is, it shouldn't force you to set --helm-binary. But I had no other way to not bloat helmfile's codebase to just add this experimental feature.

Resolves #649
Resolves #650

This enhances helmfile so that it can:

- Treat K8s manifests directories and Kustomize projects as charts
- Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (#649)
- Add adhoc patches(JSON Patch or Strategic Merge Patch supported) to be applied to the K8s resources before sync/diff/template, without forking or modifyin chart(s) (#650)

The usage is as outlined in https://github.com/mumoshu/helm-x/tree/master/examples/helmfile.

Add any or all of `dependencies:`, `jsonPatches:` and `strategicMergePatches:` so that it adds additional flags to `helm` calls that is only supported by `helm x`.

```yaml
releases:
- name: kustomize
  chart: ../kustomize
- name: manifests
  chart: ../manifests
- name: foo
  chart: incubator/raw
  dependencies:
  - alias: bar
    chart: incubator/raw
  values:
  - values.yaml
  - bar:
      enabled: true
      resources:
      - apiVersion: v1
        kind: Pod
        metadata:
          name: bar
        spec:
          containers:
          - command:
            - sleep
            - 1000
            image: alpine:3.9.4
            imagePullPolicy: IfNotPresent
            name: bar
  jsonPatches:
  - target:
      version: v1
      kind: Pod
      name: foo
    patch:
    - op: replace
      path: /spec/containers/0/command
      value:
      - sleep
      - "123"
  strategicMergePatches:
  - apiVersion: v1
    kind: Pod
    metadata:
      name: bar
    spec:
      containers:
      - name: bar
        command:
        - sleep
        - "234"
```

You can alternatively provide `source: path/to/patch.yaml` for `jsonPatches` and `strategicMergePatches` items to externalize it. Add `.gotmpl` suffix like you would do for values files for templating.

When running `helmfile` you must point `--helm-binary` to the `helm-x` binary like below:

```
$ helmfile --helm-binary ~/.helm/plugins/helm-x/bin/helm-x --log-level debug apply
```

after installing the [helm-x](https://github.com/mumoshu/helm-x) plugin.

The integration should ideally be automatic. That is, it shouldn't force you to set `--helm-binary`. But I had no other way to not bloat helmfile's codebase to just add this experimental feature.

Resolves #649
Resolves #650
@mumoshu mumoshu merged commit 1da3488 into master Jun 12, 2019
@mumoshu mumoshu deleted the helm-x branch June 12, 2019 07:11
mumoshu pushed a commit that referenced this pull request Jul 11, 2019
Use with the helm-x support(#673)

This enhances config syntax to accept adopt: [NS/KIND/RESOURCE_NAME] at the release level so that helmfile calls helm-x to transparently import existing resources at the installation time.

Resolves #84
@mlushpenko
Copy link
Contributor

mlushpenko commented Apr 14, 2020

Maybe will be useful for someone. Helm-x latest version is not working on mac, so I had to install 0.7.2 not to get any errors and to see manifest functionality working properly. And you can't use manifest directly, you have to put it in a folder and specify folder path.

Finally, 0.7.2 version doesn't seem to respect chart version and installs latest instead but it's acceptable for me for now.

To be more specific, my current setup:

$ helm version
Client: &version.Version{SemVer:"v2.16.5", GitCommit:"89bd14c1541fa93a09492010030fd3699ca65a97", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.16.5", GitCommit:"89bd14c1541fa93a09492010030fd3699ca65a97", GitTreeSt
 
$ helm plugin list
x   	0.7.2   	Turn Kubernetes manifests and Kustomization into Helm Release

Too bad, 0.7.2 has some troubles with dependency files:

identified at least one change, exiting with non-zero exit code (detailed-exitcode parameter enabled)

worker 4/4 finished
err: release "nifi" in "helmfile.yaml" failed: helm-x exited with status 255:
  I0414 23:57:54.392330   83214 chartify.go:236] using requirements.yaml:
  ---
  dependencies:
    - name: zookeeper
      version: 2.1.1
      repository: https://kubernetes-charts-incubator.storage.googleapis.com/
      condition: zookeeper.enabled
  F0414 23:57:54.465513   83214 main.go:51] exit status 1
in ./helmfile.yaml: helm-x exited with status 255:
  I0414 23:57:54.392330   83214 chartify.go:236] using requirements.yaml:
  ---
  dependencies:
    - name: zookeeper
      version: 2.1.1
      repository: https://kubernetes-charts-incubator.storage.googleapis.com/
      condition: zookeeper.enabled
  F0414 23:57:54.465513   83214 main.go:51] exit status 1

@yuri-1987
Copy link

hi @mumoshu
having issues with osx 10.15, helm 2.12.3, helmfile 0.111.0 and helm-x both 0.7.2 and 0.8.0
any idea what it could be? its not permissions, running ./helm-x works

helmfile -b /Users/yuri/helm/plugins/helm-x -e clustername -f nginx-ingress.yaml diff
panic: unexpected error: fork/exec /Users/yuri/helm/plugins/helm-x: permission denied

goroutine 1 [running]:
github.com/roboll/helmfile/pkg/helmexec.combinedOutput(0xc0004ee420, 0xc000309038, 0x14, 0x0, 0xc0004b8200, 0x14, 0x20)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/helmexec/runner.go:69 +0x44e
github.com/roboll/helmfile/pkg/helmexec.ShellRunner.Execute(0x0, 0x0, 0xc000309038, 0x7ffeefbffbcc, 0x24, 0xc00035a4e0, 0x3, 0x3, 0x0, 0x100ec58, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/helmexec/runner.go:36 +0xd9
github.com/roboll/helmfile/pkg/helmexec.detectHelm3(0x7ffeefbffbcc, 0x24, 0xc000309038, 0x26771a0, 0xc0003a0960, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/helmexec/exec.go:57 +0x109
github.com/roboll/helmfile/pkg/helmexec.New(...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/helmexec/exec.go:68
github.com/roboll/helmfile/pkg/app.(*App).getHelm(0xc0000d4240, 0xc0004fa000, 0x0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:428 +0x266
github.com/roboll/helmfile/pkg/app.(*App).ForEachState.func1(0xc0004fa000, 0xc0004fa000, 0x0, 0x0, 0xc0000e2480)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:567 +0x4a
github.com/roboll/helmfile/pkg/app.(*App).visitStates.func1(0x7ffeefbffc00, 0x12, 0xc00027acc0, 0x3e, 0x0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:517 +0x7ee
github.com/roboll/helmfile/pkg/app.(*App).visitStateFiles.func1(0x0, 0x212fa75)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:363 +0x9d
github.com/roboll/helmfile/pkg/app.(*App).within(0xc0000d4240, 0x212fa75, 0x1, 0xc00063e4c8, 0xc00063e488, 0x2)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:305 +0x602
github.com/roboll/helmfile/pkg/app.(*App).visitStateFiles(0xc0000d4240, 0x7ffeefbffc00, 0x12, 0x30a37f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:357 +0x2f2
github.com/roboll/helmfile/pkg/app.(*App).visitStates(0xc0000d4240, 0x7ffeefbffc00, 0x12, 0x30a37f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:439 +0x10b
github.com/roboll/helmfile/pkg/app.(*App).visitStatesWithSelectorsAndRemoteSupport(0xc0000d4240, 0x7ffeefbffc00, 0x12, 0xc00063e7b0, 0x0, 0x0, 0x0, 0xc0003c70a0, 0x10)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:686 +0x45c
github.com/roboll/helmfile/pkg/app.(*App).ForEachState(0xc0000d4240, 0xc00063e840, 0x0, 0x0, 0x0, 0xc0003c70a0, 0xc00063e8a8)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:566 +0xb2
github.com/roboll/helmfile/pkg/app.(*App).Diff(0xc0000d4240, 0x26a92e0, 0xc0003c70a0, 0xc0003c70a0, 0xc000309038)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:129 +0xc5
main.main.func4(0xc0000d4240, 0xc00011cc60, 0x0, 0xc0003c7090, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/main.go:214 +0x6d
main.action.func1(0xc00011cc60, 0x0, 0xc00011cc60)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/main.go:675 +0xd4
github.com/urfave/cli.HandleAction(0x1ec3b60, 0xc0003c6b40, 0xc00011cc60, 0xc0002a6900, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/.brew_home/go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0x2130aa2, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x216e2c4, 0x35, 0x0, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/.brew_home/go/pkg/mod/github.com/urfave/cli@v1.20.0/command.go:210 +0x9e8
github.com/urfave/cli.(*App).Run(0xc0000b0000, 0xc0000ca000, 0x8, 0x8, 0x0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/.brew_home/go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:255 +0x741
main.main()
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/main.go:487 +0x33e4


@mumoshu
Copy link
Collaborator Author

mumoshu commented May 27, 2020

@yuri-1987 Hey! Reading your logs, I guess you should use /Users/yuri/helm/plugins/helm-x/bin/helm-x

mumoshu added a commit that referenced this pull request May 27, 2020
This is the GA version of the helm-x integration #673 developed last year.

You get all the following benefits without an extra helm plugin:

- Ability to add ad-hoc chart dependencies/aliases, without forking the chart (Fixes #876 )
- Ability to patch resulting K8s resources before installing the helm chart
- Ability to install a kustomization as a chart (Requires `kustomize` binary to be available in `$PATH`
- Ability to install a directory of K8s manifests as a chart
- etc.
@yuri-1987
Copy link

yuri-1987 commented May 27, 2020

thanks @mumoshu, i moved to helmfile 0.118.0
with that i'm getting F0527 09:09:14.741081 35123 main.go:51] unknown flag: --kube-context at v0.7.2
with v0.8.0 i am getting another error:

helmfile -b /Users/yuri/helm/plugins/helm-x/bin/helm-x -e clustername -f nginx-ingress.yaml diff
panic: helm-x exited with status 2:
  panic: exec: "": executable file not found in $PATH

  goroutine 1 [running]:
  github.com/mumoshu/helm-x/pkg/helmx.(*Runner).IsHelm3(0xc0004cace0, 0x8)
  	/home/circleci/project/pkg/helmx/helm3.go:21 +0x19a
  main.main()
  	/home/circleci/project/main.go:31 +0x73

goroutine 1 [running]:
github.com/roboll/helmfile/pkg/helmexec.detectHelm3(0x7ffeefbff6dc, 0x2f, 0xc00038d288, 0x26771a0, 0xc0000ba8a0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/helmexec/exec.go:59 +0x1a6
github.com/roboll/helmfile/pkg/helmexec.New(...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/helmexec/exec.go:68
github.com/roboll/helmfile/pkg/app.(*App).getHelm(0xc0002cc900, 0xc0000d0fc0, 0x0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:428 +0x266
github.com/roboll/helmfile/pkg/app.(*App).ForEachState.func1(0xc0000d0fc0, 0xc0000d0fc0, 0x0, 0x0, 0xc0000e1b00)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:567 +0x4a
github.com/roboll/helmfile/pkg/app.(*App).visitStates.func1(0x7ffeefbff71b, 0x12, 0xc000319280, 0x3e, 0x0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:517 +0x7ee
github.com/roboll/helmfile/pkg/app.(*App).visitStateFiles.func1(0x0, 0x212fa75)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:363 +0x9d
github.com/roboll/helmfile/pkg/app.(*App).within(0xc0002cc900, 0x212fa75, 0x1, 0xc0006be4c8, 0xc0006be488, 0x2)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:305 +0x602
github.com/roboll/helmfile/pkg/app.(*App).visitStateFiles(0xc0002cc900, 0x7ffeefbff71b, 0x12, 0x30a37f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:357 +0x2f2
github.com/roboll/helmfile/pkg/app.(*App).visitStates(0xc0002cc900, 0x7ffeefbff71b, 0x12, 0x30a37f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:439 +0x10b
github.com/roboll/helmfile/pkg/app.(*App).visitStatesWithSelectorsAndRemoteSupport(0xc0002cc900, 0x7ffeefbff71b, 0x12, 0xc0006be7b0, 0x0, 0x0, 0x0, 0xc0004c1650, 0x10)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:686 +0x45c
github.com/roboll/helmfile/pkg/app.(*App).ForEachState(0xc0002cc900, 0xc0006be840, 0x0, 0x0, 0x0, 0xc0004c1650, 0xc0006be8a8)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:566 +0xb2
github.com/roboll/helmfile/pkg/app.(*App).Diff(0xc0002cc900, 0x26a92e0, 0xc0004c1650, 0xc0004c1650, 0xc00038d288)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/pkg/app/app.go:129 +0xc5
main.main.func4(0xc0002cc900, 0xc0001929a0, 0x0, 0xc0004c1640, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/main.go:214 +0x6d
main.action.func1(0xc0001929a0, 0x0, 0xc0001929a0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/main.go:675 +0xd4
github.com/urfave/cli.HandleAction(0x1ec3b60, 0xc0004c1100, 0xc0001929a0, 0xc00032d000, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/.brew_home/go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0x2130aa2, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x216e2c4, 0x35, 0x0, ...)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/.brew_home/go/pkg/mod/github.com/urfave/cli@v1.20.0/command.go:210 +0x9e8
github.com/urfave/cli.(*App).Run(0xc0000b0000, 0xc0000c8000, 0x8, 0x8, 0x0, 0x0)
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/.brew_home/go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:255 +0x741
main.main()
	/private/tmp/helmfile-20200418-70486-15lpuib/helmfile-0.111.0/main.go:487 +0x33e4

@msutter
Copy link
Contributor

msutter commented May 27, 2020

Hi guys,
same issue on osx here with helm v3.2.0, helmfile version v0.118.1, and helm-x version 0.8.0.
I'm on osx 10.13

macbook-pro-3:0.8.0 tgdsuma5$ ./helm-x
panic: exec: "": executable file not found in $PATH

goroutine 1 [running]:
github.com/mumoshu/helm-x/pkg/helmx.(*Runner).IsHelm3(0xc00009e960, 0x8)
	/home/circleci/project/pkg/helmx/helm3.go:21 +0x19a
main.main()
	/home/circleci/project/main.go:31 +0x73

@yuri-1987
Copy link

hi @msutter, afaik from version 0.118.0 helm-x functionality is already inside helmfile's code, can you try to run it without the --helm-binary flag?

@msutter
Copy link
Contributor

msutter commented May 28, 2020

Thanks @yuri-1987, I didn't notice the integration.
Works fine now.

@mumoshu any plans to integrate more kustomize transformers like commonLabels & commonAnnotations ?

By the way thanks a lot for this great piece of software.

@mumoshu
Copy link
Collaborator Author

mumoshu commented May 29, 2020

@msutter Not yet, but I believe they worth dedicated feature requests to Helmfiel!

Would you mind creating those issues, so that I won't forget adding them?

@schue
Copy link

schue commented Aug 28, 2020

It seems like helmfile is completely ignoring the contents of the jsonPatches section for me. It sees that section because if I misspell jsonPatches it errors out but the content of that section is ignored. For instance:

jsonPatches:                                                                                              
- targt:                                                                                                 
  weasels: flop                                                                                           

does not provoke a complaint and has no effect. Valid patches content is also ignored.
helmfile version v0.125.7
helm: 3.3.0

I'm trying to modify loose manifests, does it only work for helm charts?

@mumoshu
Copy link
Collaborator Author

mumoshu commented Aug 28, 2020

does it only work for helm charts?

Yes. But that's being fixed via #1437

@4c74356b41
Copy link

hm, when is the fix released? @mumoshu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: patching chart without forking it feat: adhoc chart dependencies
6 participants