[WIP] Ironic service #93

mateusz-blaszkowski · 2016-09-30T07:24:19Z

No description provided.

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Not verified yet Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

PXE is set up temporarily. Replace it with iPXE in the future. Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

DTadrzak · 2016-10-05T08:01:50Z

ironic/manifest.jsonnet

+      replicas: 1,
+
+      image: {
+        base: "quay.io/stackanetes/stackanetes-%s:barcelona",


can add a new line between base and init image?

DTadrzak · 2016-10-05T08:05:41Z

ironic/templates/compute/daemonset.yaml.j2

+              mountPath: /configmaps/nova-compute-ironic.sh
+            - name: resolvconf
+              mountPath: /configmaps/resolv.conf
+            - mountPath: /var/lib/nova


Can you switch the order?

DTadrzak · 2016-10-05T08:05:59Z

ironic/templates/compute/daemonset.yaml.j2

+            name: ironic-resolvconf
+        - name: varlibnovaironic
+          hostPath:
+            path: /var/lib/nova/ironic


DTadrzak · 2016-10-05T08:06:20Z

ironic/templates/conductor/conductor.yaml.j2

+      nodeSelector:
+        {{ deployment.control_node_label }}: enabled
+      # TODO(DTadrzak): it must be removed in future
+      securityContext:


Can we remove it?

agree we should just add in in tftpd container.

DTadrzak · 2016-10-05T08:06:54Z

ironic/templates/conductor/conductor.yaml.j2

+          image: {{ deployment.image.conductor }}
+          imagePullPolicy: Always
+          # TODO(DTadrzak): it must be removed in future
+          securityContext:


DTadrzak · 2016-10-05T08:16:46Z

ironic/templates/configmaps/nova.conf.yaml.j2

+    security_group_api = neutron
+    network_api_class = nova.network.neutronv2.api.API
+
+    linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver


linuxnet_interface_driver = openvswitch

we should use none or noop here, we don't need openvswitch here.

DTadrzak · 2016-10-05T08:17:32Z

ironic/templates/configmaps/start.sh.yaml.j2

+    #!/bin/bash
+    set -ex
+
+    ironic-api


DTadrzak · 2016-10-05T08:18:28Z

ironic/templates/jobs/post.yaml.j2

+              value: "keystone-api"
+            - name: DEPENDENCY_CONFIG
+              value: "/tmp/post.sh"
+            - name: ANSIBLE_LIBRARY


You dont need it.

DTadrzak · 2016-10-05T08:19:10Z

neutron/templates/configmaps/ml2-conf.ini.yaml.j2

@@ -19,9 +25,14 @@ data:
    vxlan_group = 239.1.1.1

    [securitygroup]
+    {% if ironic.enabled %}
+    firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver


make sure that neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver is still valid.

for ironic it should be noop.

DTadrzak · 2016-10-05T08:20:05Z

nova/templates/configmaps/nova.conf.yaml.j2

@@ -29,13 +27,40 @@ data:
    use_neutron = True
    security_group_api = neutron
    network_api_class = nova.network.neutronv2.api.API
-    # TODO(DTadrzak): FIX ME
-    firewall_driver = nova.virt.firewall.NoopFirewallDriver
+
    linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver


linuxnet_interface_driver = openvswitch

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

ss7pro · 2016-10-12T11:41:32Z

ironic/templates/api/deployment.yaml.j2

+    spec:
+      # TODO(DTadrzak): it must be removed in future
+      securityContext:
+        runAsUser: 0


please drop it.

ss7pro · 2016-10-12T11:41:59Z

ironic/templates/compute/daemonset.yaml.j2

+        # {{ deployment.compute_node_label }}: enabled
+        mateuszb: enabled
+      securityContext:
+        runAsUser: 0


remove that.

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

ss7pro · 2016-10-18T10:28:05Z

glance/manifest.jsonnet

@@ -81,6 +81,13 @@ kpm.package({
      glance_keyring: "",
    },

+    swift: {
+      enabled: false,
+      auth_address: "http://10.91.62.52:80/auth/1.0",


we should change it to rgw/auth/1.0 and have radosgw deployed on kubernetes. Some time ago we had a radosgw in stackanetes so we may reuse it.

ss7pro · 2016-10-18T10:29:01Z

ironic/manifest.jsonnet

+  variables: {
+    deployment: {
+      control_node_label: "openstack-control-plane",
+      compute_node_label: "openstack-ironic-compute-node",


This is not really needed, openstack computes for Ironic are fully stateless for now.

ss7pro · 2016-10-18T10:29:39Z

ironic/manifest.jsonnet

+        init: $.variables.deployment.image.base % "kolla-toolbox",
+        post: $.variables.deployment.image.base % "kolla-toolbox",
+        # TODO:
+        api: "10.91.96.87:5000/debian-source-ironic-api:mateuszb-ironic",


we should push those image to quay.io

ss7pro · 2016-10-18T10:32:42Z

ironic/templates/compute/daemonset.yaml.j2

@@ -0,0 +1,59 @@
+apiVersion: extensions/v1beta1
+kind: DaemonSet


nova compute should be a DaemonSet or I would even opt to make it a PetSet (consitent hostnames).

It is a DaemonSet now - should I change it into a PetSet?

yes, PetSet.

@ss7pro PetSet is currently not supported in kpm. I'll make it a Deployment for now

ss7pro · 2016-10-18T10:33:08Z

ironic/templates/compute/daemonset.yaml.j2

+    spec:
+      nodeSelector:
+        # TODO:
+        # {{ deployment.compute_node_label }}: enabled


it should use control plane label.

ss7pro · 2016-10-18T10:33:29Z

ironic/templates/compute/daemonset.yaml.j2

+          image: {{ deployment.image.compute }}
+          imagePullPolicy: Always
+          securityContext:
+            privileged: true


not needed.

ss7pro · 2016-10-18T10:34:07Z

ironic/templates/compute/daemonset.yaml.j2

+            - name: DEPENDENCY_SERVICE
+              value: "keystone-api,glance-api,nova-api"
+            - name: DEPENDENCY_CONFIG
+              value: "/etc/nova/nova.conf,/etc/resolv.conf"


can we mount /etc/resolv.conf as a configmap, as done here: #99

ss7pro · 2016-10-18T10:34:30Z

ironic/templates/compute/daemonset.yaml.j2

+          configMap:
+            name: ironic-resolvconf
+        - name: varlibnovaironic
+          hostPath:


this should be an emptyDir.

ss7pro · 2016-10-18T10:35:50Z

ironic/templates/conductor/conductor.yaml.j2

+            - name: DEPENDENCY_SERVICE
+              value: "rabbitmq"
+            - name: DEPENDENCY_CONFIG
+              value: "/etc/ironic/ironic.conf,/etc/resolv.conf"


same thing for /etc/resolv.conf we should use it as here: #99

ss7pro · 2016-10-18T10:36:38Z

ironic/templates/conductor/conductor.yaml.j2

+          image: {{ deployment.image.ironic_http }}
+          imagePullPolicy: Always
+          # TODO(DTadrzak): it must be removed in future
+          securityContext:


this can be removed.

ss7pro · 2016-10-18T10:38:34Z

ironic/templates/configmaps/ironic.conf.yaml.j2

+    enabled_drivers = pxe_ipmitool,agent_ipmitool_socat,agent_pyghmi,fake
+
+    [api]
+    api_workers=128


this should be parametrized: {{ misc.workers }} and should be 8 by default.

ss7pro · 2016-10-18T10:39:36Z

ironic/templates/configmaps/ironic.conf.yaml.j2

+
+    [database]
+    connection = mysql+pymysql://{{ database.ironic_user }}:{{ database.ironic_password }}@{{ database.address }}/{{ database.ironic_database_name }}
+


add max_retries = -1

ss7pro · 2016-10-18T10:41:12Z

ironic/templates/configmaps/nginx-default.conf.yaml.j2

+apiVersion: v1
+kind: ConfigMap
+data:
+  default.conf: |+


I would also redirect access and error log to std{out,err}:
error_log /dev/fd/2 info;

http {
access_log /dev/fd/1;
...
}

it's done by default in nginx:alpine :

/ # ls -lah /var/log/nginx/error.log lrwxrwxrwx 1 root root 11 Sep 23 17:11 /var/log/nginx/error.log -> /dev/stderr / # ls -lah /var/log/nginx/access.log lrwxrwxrwx 1 root root 11 Sep 23 17:11 /var/log/nginx/access.log -> /dev/stdout

let me know if this is not the solution you expect

ss7pro · 2016-10-18T10:41:38Z

ironic/templates/configmaps/nova.conf.yaml.j2

+  nova.conf: |+
+    [DEFAULT]
+    debug = {{ misc.debug }}
+    # TODO(DTadrzak): FIX ME


vif related stuff should be dropped.

@ss7pro I believe that we don't want to create dozens of conditionals. Common nova.conf will be used by nova-compute, ironic-compute, nova-api, nova-conductor, nova-scheduler, ... If we want to hide some config options for specific services, we'll end up with multiple dirty if-elif-else statements

ss7pro · 2016-10-18T10:42:51Z

ironic/templates/configmaps/nova.conf.yaml.j2

+    connection = mysql+pymysql://{{ database.nova_user }}:{{ database.nova_password }}@{{ database.address }}/{{ database.nova_api_database_name }}
+    max_pool_size = {{ misc.workers }}
+    max_overflow  = {{ misc.workers }}
+


max_retries = -1

ss7pro · 2016-10-18T10:43:01Z

ironic/templates/configmaps/nova.conf.yaml.j2

+    username = {{ keystone.neutron_user }}
+    password = {{ keystone.neutron_password }}
+
+    [database]


we should remove this section.

thereallukl · 2016-10-18T10:30:39Z

glance/templates/api/configmaps/glance-api.conf.yaml.j2

    stores = file, http, rbd
    default_store = rbd
    rbd_store_pool = {{ ceph.glance_pool }}
    rbd_store_user = {{ ceph.glance_user }}
    rbd_store_ceph_conf = /etc/ceph/ceph.conf
    rbd_store_chunk_size = 8
+
+    {% elif swift.enabled -%}
+    stores = file, http, swift


swift and rbd should not be exclusive, please allow having both stores enabled

what should be the default store in such case?

I believe it calls for an additional flag (with swift generally being ok as default).

thereallukl · 2016-10-18T10:32:58Z

ironic/manifest.jsonnet

+        init: $.variables.deployment.image.base % "kolla-toolbox",
+        post: $.variables.deployment.image.base % "kolla-toolbox",
+        # TODO:
+        api: "10.91.96.87:5000/debian-source-ironic-api:mateuszb-ironic",


should default to quay.io

thereallukl · 2016-10-18T10:38:57Z

ironic/templates/compute/daemonset.yaml.j2

+          image: {{ deployment.image.compute }}
+          imagePullPolicy: Always
+          securityContext:
+            privileged: true


what do you need privileged mode for?

thereallukl · 2016-10-18T10:43:20Z

ironic/manifest.jsonnet

+      api_url: "http://%s:6385/v1" % $.variables.network.ip_address,
+
+      dns:  {
+        servers: ["10.3.0.10"],


I believe it should use global variable.

See #93 (comment)

thereallukl · 2016-10-18T10:43:27Z

ironic/manifest.jsonnet

+
+      dns:  {
+        servers: ["10.3.0.10"],
+        kubernetes_domain: "cluster.local",


See #93 (comment)

thereallukl · 2016-10-18T10:44:18Z

ironic/manifest.jsonnet

+        conductor: "10.91.96.87:5000/debian-source-ironic-conductor:mateuszb-ironic",
+        compute: "10.91.96.87:5000/debian-source-nova-compute-ironic:mateuszb-ironic",
+        ironic_pxe: "10.91.96.87:5000/debian-source-ironic-pxe:mateuszb-ironic",
+        ironic_http: "nginx:alpine",


should be debian based image

Agreed to leave public nginx image - i'll just use more precise (nginx:1.11.5-alpine) version

ss7pro · 2016-10-18T10:50:22Z

neutron/templates/configmaps/ml2-conf.ini.yaml.j2

+    {% if ironic.enabled %}
+    type_drivers = flat
+    tenant_network_types = flat
+    mechanism_drivers = openvswitch


we should have noop or empty line here.

ss7pro · 2016-10-18T10:52:24Z

nova/templates/configmaps/nova.conf.yaml.j2

@@ -8,8 +8,6 @@ data:
    vif_plugging_is_fatal = False


Why are we having multiple nova.conf templates for ironic ? Can we have a single common one ?

I'll make a single one with conditional configuration block when ironic_compute is enabled

thereallukl · 2016-10-18T10:54:31Z

glance/templates/api/configmaps/glance-api.conf.yaml.j2

+    default_store = swift
+    swift_store_auth_version = 1
+    swift_store_auth_address = {{ swift.auth_address }}
+    swift_store_user = {{ swift.glance_user }}


s/glance_user/store_user/

thereallukl · 2016-10-18T10:55:39Z

ironic/manifest.jsonnet

+
+      ingress: {
+        enabled: true,
+        host: "%s.openstack.cluster",


take value for host from parameters.yaml

Values from the manifest are overwritten with the same values from parameters.yaml (if present). However, they need to be specified in a manifest.

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

auvipy

could you please rebase?

thereallukl · 2019-05-18T08:36:15Z

@auvipy this project is dead. If you're interested in running openstack on k8s please take a look at openstack-helm project (https://github.com/openstack/openstack-helm)

mateusz-blaszkowski added 10 commits September 27, 2016 14:09

Ironic: init job

efb0704

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Ironic: post job

a29d1bd

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

[WIP] Ironic-API

1c67f73

Not verified yet Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Ironic db-sync job

6c3ac7a

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Ironic-conductor service

f6b2bec

Nova: configurable compute driver (Libvirt/Ironic)

0bb5c46

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Nova-compute-ironic service

d8fd2e7

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Nova-compute configuration fix

720bbff

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Ironic: configure networking to communicate with the bare metal server

e2dfd18

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Ironic: Ironic-pxe container added to ironic-conductor POD

eacd77c

PXE is set up temporarily. Replace it with iPXE in the future. Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

DTadrzak reviewed Oct 5, 2016

View reviewed changes

mateusz-blaszkowski added 3 commits October 6, 2016 10:55

Review changes.

be6633e

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Glance: support for Swift backend (required by agent_* Ironic drivers)

0c0dd2b

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Openvswitch: add host port to bridge when Ironic is enabled

377c82f

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

ss7pro reviewed Oct 12, 2016

View reviewed changes

mateusz-blaszkowski added 2 commits October 14, 2016 13:57

iPXE support

4a08650

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Serial console enabled

8813fb0

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

ss7pro reviewed Oct 18, 2016

View reviewed changes

thereallukl reviewed Oct 18, 2016

View reviewed changes

ss7pro reviewed Oct 18, 2016

View reviewed changes

thereallukl reviewed Oct 18, 2016

View reviewed changes

mateusz-blaszkowski added 6 commits October 18, 2016 13:06

Merge remote-tracking branch 'origin/master' into ironic

22e20d0

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Addressing review comments

b7db665

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Addressing review comments

df4c22a

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Ironic images pushed to quay.io

51bd1f4

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Baremetal provisioning configuration options tweaked

7eb6f8d

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

Merge remote-tracking branch 'origin/master' into ironic

4f4fcae

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

mateusz-blaszkowski mentioned this pull request Oct 21, 2016

Rados gateway (dependency for Ironic agent_* drivers) #120

Open

Making Ironic configuration more consistent between services

0ea299a

Signed-off-by: Mateusz Blaszkowski <mateusz.blaszkowski@intel.com>

auvipy suggested changes May 17, 2019

View reviewed changes

		@@ -0,0 +1,59 @@
		apiVersion: extensions/v1beta1
		kind: DaemonSet


		[database]
		connection = mysql+pymysql://{{ database.ironic_user }}:{{ database.ironic_password }}@{{ database.address }}/{{ database.ironic_database_name }}

[WIP] Ironic service #93

Are you sure you want to change the base?

[WIP] Ironic service #93

Conversation

mateusz-blaszkowski commented Sep 30, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ss7pro Oct 18, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mateusz-blaszkowski Oct 19, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thereallukl Oct 18, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

auvipy left a comment

Choose a reason for hiding this comment

thereallukl commented May 18, 2019

ss7pro Oct 18, 2016 •

edited

mateusz-blaszkowski Oct 19, 2016 •

edited

thereallukl Oct 18, 2016 •

edited