SMOLNET PORTAL home about changes
[root][parent][cert][raw file]
=====================================================================
             ___       __  _ __
  ___  _____/ (_)___  / /_(_) /__
 / _ \/ ___/ / / __ \/ __/ / //_/
/  __/ /__/ / / /_/ / /_/ / ,<
\___/\___/_/_/ .___/\__/_/_/|_|
            /_/
=====================================================================


ACI Connector for k8s on a Raspberry Pi Cluster

2017-09-12 | #raspberrypi #kubernetes #arm #azure #docker


Intro


Running the Azure Container Instances Connector for Kubernetes[1] on a Raspberry Pi Cluster.

1: https://web.archive.org/web/20171104111735/https://github.com/azure/aci-connector-k8s (https://web.archive.org)


One of the most interesting features of Azure Container Instances[2] is the Azure Container Instances Connector for Kubernetes[3]. This adds an ACI "node" to an existing Kubernetes[4] cluster and allows you to deploy pods to it. This "node" will run pods in ACI without having to create or manage and additional Azure VMs, just point-and-shoot a pod at it and it will run with no additional setup required.

2: https://docs.microsoft.com/en-us/azure/container-instances/ (https://docs.microsoft.com)
3: https://web.archive.org/web/20171104111735/https://github.com/azure/aci-connector-k8s (https://web.archive.org)
4: https://kubernetes.io (https://kubernetes.io)


By using the ACI Connector for Kubernetes on a Raspberry PI, a cluster can run homogenous ARM containers on-prem, but still have the ability to deploy and manage x86 containers to a cloud provider.

Read more about Azure Container Instances,

  • https://thenewstack.io/azure-container-instances-mean-cheaper-agile-container-tools-way/[5]
  • https://docs.microsoft.com/en-us/azure/container-instances/[6]
  • https://azure.microsoft.com/en-us/blog/announcing-azure-container-instances/[7]


5: https://thenewstack.io/azure-container-instances-mean-cheaper-agile-container-tools-way/ (https://thenewstack.io)
6: https://docs.microsoft.com/en-us/azure/container-instances/ (https://docs.microsoft.com)
7: https://azure.microsoft.com/en-us/blog/announcing-azure-container-instances/ (https://azure.microsoft.com)


Creating an aci-connector-k8s ARM Image for Raspberry Pi


The upstream aci-connector-k8s image is x86 only, but since it's written in typescript it can easily be run on different architectures. To run on a Raspberry Pi k8s cluster, all that is required is building an armhf Docker image.

Building a Nodejs ARM Docker Image


Note: As of 9/12/2017 Docker Hub Official Images support multi-platform[8] and re-building an image for armhf (or arm64, ppc64le, and s390x) is no longer required if using a supported image (currently Debian based only, Alpine based like below still need to be re-built).


8: https://integratedcode.us/2017/09/13/dockerhub-official-images-go-multi-platform/ (https://integratedcode.us)


The aci-connector-k8s Dockerfile[9] uses `node:8.4.0-alpine` as it's base image. While there are some "unofficial" node ARM images, lets create one from a somewhat official repository. This involves finding an armhf Alpine 3.6 image, copying/pasting the node:8.4-alpine Dockerfile[10], replacing the `FROM` with the armhf version of alpine, and building the image.

9: https://web.archive.org/web/20211229050703/https://github.com/Azure/aci-connector-k8s/blob/master/Dockerfile (https://web.archive.org)
10: https://github.com/nodejs/docker-node/blob/17c50cb300581280805a4183524fbf57840f3a7e/8.4/alpine/Dockerfile (https://github.com)


There are two ways to build an armhf image,

  • Build on Raspberry Pi


* 100% native build, less likely to run into bugs
* Painfully slow due to Raspberry Pi hardware

  • Cross-build using Docker for Mac[11] or Multiarch[12]


* Dramatically speed up builds
* Could have unforseen issues due to running in an emulated environment

11: https://docs.docker.com/docker-for-mac/multi-arch/ (https://docs.docker.com)
12: https://github.com/multiarch (https://github.com)




After first attempting option 1, two hours later and losing the ability to ssh into the Raspberry Pi, option 2 is a much faster approach. Building on a MacBook Pro using the built-in multiarch features of Docker for Mac works well, but is still slow even on a 4 core system. Fortunately Using up a 24-core Packet.net[13] Type 2 bare-metal instance to cross-compile using Multiarch is easy to do too.

13: https://www.packet.net (https://www.packet.net)


Note: The official Docker images for armhf are arm32v6[14] and arm32v7[15]. These will work natively on a Raspberry Pi, Docker for Mac, Multiarch, and a Linux system with `qemu-*-static support`. For full details on these see my post on cross-building Docker images[16].


14: https://hub.docker.com/r/arm32v6/ (https://hub.docker.com)
15: https://hub.docker.com/r/arm32v7/ (https://hub.docker.com)
16: http://www.ecliptik.com/Cross-Building-and-Running-Multi-Arch-Docker-Images/ (http://www.ecliptik.com)


A armhf multiarch nodejs Dockerfile[17] was built on the Packet.net Type 2 instance and pushed to Docker hub as `ecliptik/node:8.4.0-alpine-armhf`. This only took a few minutes using the Type 2 instance, much faster than Raspberry Pi or Macbook Pro.

17: https://github.com/ecliptik/dockerfiles/blob/master/node/Dockerfile.alpine.armhf (https://github.com)


Example Using Multiarch to re-build a node armhf alpine image,

curl -sSL https://raw.githubusercontent.com/nodejs/docker-node/c044d61e6d02756bb8ed1557b2f0c7a0d7fead6f/8.4/alpine/Dockerfile | sed "s/alpine:3.6/multiarch\/alpine:armhf-v3.6/" > Dockerfile.node.armhf

docker build -f Dockerfile.node.armhf -t ecliptik/node:8.4.0-alpine-armhf .
docker push ecliptik/node:8.4.0-alpine-armhf


Building an aci-connector-k8s ARM Docker Image


Once a nodejs arm-alpine image is created, clone the aci-connector-k8s[18] repositoriy, and update the `Dockerfile` to use the `ecliptik/node:8.4.0-alpine-armhf` image. Additionaly, use the `Dockefile` below to use multi-stage builds for improved image size.

18: https://web.archive.org/web/20171104111735/https://github.com/azure/aci-connector-k8s (https://web.archive.org)


### Base Image
# Setup up a base image to use in Build and Runtime images
FROM ecliptik/node:8.4.0-alpine-armhf AS base

WORKDIR /app
COPY package.json .

### Build Image
# Installs build dependencies and npm packages
# Creates artifacts to copy into Runtime image
FROM base AS build

# Install build OS packages
RUN set -ex && \
        buildDeps=' \
                make \
                gcc \
                g++ \
                python \
                py-pip \
                curl \
                openssl \
        ' && \
    apk add --no-cache \
       --virtual .build-deps $buildDeps

#Copy application into build image
COPY . .

# Install npm packages
RUN npm install -g
RUN npm install --silent --save-dev -g \
       gulp-cli \
       typescript

# Compile typescript sources to javascript artifacts
RUN tsc --target es5 connector.ts

### Runtime Image
# Copy artifacts from Build image and setups up entrypoint/cmd to run app
FROM base AS runtime

# Copy artifacts from Build Image
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/*.js ./
COPY --from=build /app/LICENSE ./

# Runtime command
ENTRYPOINT ["node"]
CMD ["connector.js"]


With an updated `Dockerfile` in the cloned repo, build the aci-connector-k8s image *on* a Raspberry Pi,

docker build -t ecliptik/aci-connector-k8s:alpine-armhf .
docker push ecliptik/aci-connector-k8s:alpine-armhf


Note: Trying to build the image on non-native armhf platform like Docker for Mac or Multiarch may result in errors like `"SyntaxError: Unexpected end of JSON input"`. The image only seems to build on native Raspberry Pi or ARM hardware.


Running the ACI Connector


Now that we have a armhf image capable of running on a Raspberry Pi, we can deploy the pod to a Raspberry Pi Kubernetes cluster[19].

19: http://www.ecliptik.com/Raspberry-Pi-Kubernetes-Cluster/ (http://www.ecliptik.com)


First clone the aci-connector-k8s[20] repository onto the Raspberry Pi cluster master,

20: https://github.com/Azure/aci-connector-k8s (https://github.com)


git clone https://github.com/Azure/aci-connector-k8s.git


Edit the `examples/aci-connector.yaml` and update the `image` to use the `ecliptik/aci-connector-k8s:alpine-armhf` image.

Next, if you used `kubeadm` to create your cluster and RBAC is enabled, you'll need to create a role and set it up for the connector. This is discussed in this Github issue[21] that includes creating a RBAC role and updating the service to use it.

21: https://github.com/Azure/aci-connector-k8s/issues/26#issuecomment-326809041 (https://github.com)


Create the RBAC role for the connector,

curl -sSL https://raw.githubusercontent.com/alexjmoore/aci-connector-k8s-arm/master/aci-connector-rbac.yaml | kubectl create -f -


Under `spec` in the `examples/aci-connector.yaml` add the RBAC role,

serviceAccountName: aci-connector-sa


Finally after the connector is setup to use the armhf image and RBAC, follow the rest of the *Quickstart* guide in the aci-connector-k8s README[22] to set up everything else required to run the connector (Azure keys, deployment of service, etc).

22: https://github.com/Azure/aci-connector-k8s/blob/master/README.md (https://github.com)


Working Example


Updated `examples/aci-connector.yaml` with RBAC role and `ecliptik/aci-connector-k8s:alpine-armhf` image

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: aci-connector
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: aci-connector
    spec:
      serviceAccountName: aci-connector-sa
      containers:
      - name: aci-connector
        image: ecliptik/aci-connector-k8s:alpine-armhf
        imagePullPolicy: Always
        env:
        - name: AZURE_CLIENT_ID
          value: 00000-000-00000-0000-0000
        - name: AZURE_CLIENT_KEY
          value: 00000-000-00000-0000-0000
        - name: AZURE_TENANT_ID
          value: 00000-000-00000-0000-0000
        - name: AZURE_SUBSCRIPTION_ID
          value: 100000-000-00000-0000-0000
        - name: ACI_RESOURCE_GROUP
          value: aci-test


Deploy the aci-connector pod,

kubectl create -f examples/aci-connector.yaml


Wait a few minutes while the pod comes into service (mostly waiting for the image to pull) on a worker node.

Verify aci-connector pod has started,

kubectl get pods
NAME                             READY     STATUS    RESTARTS   AGE
aci-connector-1252680567-b88w6   1/1       Running   0          3m


Verify aci-connector node is added,

kubectl get nodes -o wide
NAME            STATUS    AGE       VERSION   EXTERNAL-IP   OS-IMAGE                        KERNEL-VERSION
aci-connector   Ready     2m        v1.6.6    <none>        <unknown>                       <unknown>
navi            Ready     1h        v1.7.5    <none>        Raspbian GNU/Linux 8 (jessie)   4.4.50-hypriotos-v7+
tael            Ready     1h        v1.7.5    <none>        Raspbian GNU/Linux 8 (jessie)   4.4.50-hypriotos-v7+
tatl            Ready     1h        v1.7.5    <none>        Raspbian GNU/Linux 8 (jessie)   4.4.50-hypriotos-v7+


Deploy `example/nginx-pod.yaml` pod from aci-connector-k8s repo,

kubectl create -f examples/nginx-pod.yaml
pod "nginx" created


Verify pod deployed and is running in ACI,

kubectl get pods -o wide
 NAME                             READY     STATUS    RESTARTS   AGE       IP               NODE
 aci-connector-1696751608-tcjcq   1/1       Running   0          24m       10.244.2.4       tael
 nginx                            1/1       Running   0          10s       104.42.235.280   aci-connector



While researching and setting this up I came across many good resources on running Docker on ARM,

  • https://help.packet.net/armv8/docker-on-armv8[23]
  • https://github.com/docker-library/official-images#architectures-other-than-amd64[24]
  • https://blog.hypriot.com/post/first-touch-down-with-docker-for-mac/[25]
  • https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/[26]


23: https://help.packet.net/armv8/docker-on-armv8 (https://help.packet.net)
24: https://github.com/docker-library/official-images#architectures-other-than-amd64 (https://github.com)
25: https://blog.hypriot.com/post/first-touch-down-with-docker-for-mac/ (https://blog.hypriot.com)
26: https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/ (https://resin.io)


ARM Docker Image Repositories

  • arm32v6[27] - Works on RPI 3 with HypriotOS[28] 32-bit
  • arm32v7[29] - Works on RPI 3 with HypriotOS[30] 32-bit
  • arm64v8[31] - Currently no 64-bit version of HypriotOS, but will work on Packet.net Type 2A Instances[32]


27: https://hub.docker.com/u/arm32v6/ (https://hub.docker.com)
28: https://blog.hypriot.com/downloads/ (https://blog.hypriot.com)
29: https://hub.docker.com/u/arm32v7/ (https://hub.docker.com)
30: https://blog.hypriot.com/downloads/ (https://blog.hypriot.com)
31: https://hub.docker.com/u/arm64v8/ (https://hub.docker.com)
32: https://www.packet.net/bare-metal/servers/type-2a/ (https://www.packet.net)


Tags


docker
azure
arm
kubernetes
raspberrypi

____________________________________________________________________

Home
Response: 20 (Success), text/gemini
Original URLgemini://rawtext.club/~ecliptik/_posts/2017-09-12-ACI-Con...
Status Code20 (Success)
Content-Typetext/gemini; charset=utf-8; lang=en