I dockerized everything!

[crazyquark]

I finally dockerized my entire setup && just wanted to share.

Screenshot_2019-12-16 Portainer.png (76.5 KiB) Viewed 2361 times

The XU4 Cloudshell is running my website and media related stuff: emby, sonarr, radarr, jackett etc.

Screenshot_2019-12-16 Portainer(1).png (149.85 KiB) Viewed 2361 times

The HC1 is running Portainer which also manages the XU4 endpoint plus Home Assistant, LibreNMS and the FreePBX I mentioned in another post.

Screenshot_2019-12-16 Portainer(2).png (158.95 KiB) Viewed 2361 times

Humm, now I am thinking I should've used Kubernetes...

[mad_ady]

Congratulations!
I wonder if you have multiple web interfaces that you want to expose as port 80 and you have to use some sort of ngnix proxy to serve between them...

Regarding kubernetes that would only add complexity - some of your apps have state (e.g. the dbs) so you either need to expose the db files on a shared storage (but then network would be a bottleneck), or you'd need to add labels and restrict containers to a host.
Docker swarm might be lighter and would have the advantage of overlay network that would expose all servives on all hosts. But that is recommended from 3 nodes onwards.

I wonder how ram usage is like, compared to the traditional (apt-get from the distro) method. I fear it's worse, but I have no idea by how much.

Also - what was your motivation to run like this? Just curiosity, or do you run newer versions than the distro offers?

[crazyquark]

Thanks!
Ok, so motivation: my emby kept crashing and I found it was slightly more stable in a docker. Also, in the past I messed up my OS and had to reinstall stuff by hand; now, all I have to do is docker-compose up some stuff and my environment is back up in minutes and its OS independent(sort of) - assuming the disks themselves don't get corrupted. Also, I found this source of nice ARM images https://www.linuxserver.io/ which provides almost everything I need - in the past, having to build ARM images from Dockerfiles had been a deterrent against using docker in my setup. So, long story short, I wanted an easy to reproduce environment should things go south.

Also, I spent a lot of time setting up the FreePBX docker because I wanted to get rid of the dedicated raspberry pi - this is the second point, having a multi appliance system, where each service can have its own dependencies and quirks.

And third - yes newer versions - in combination with watchtower, the apps stay up to date automatically as newer images are being pulled when they become available - similar I think with how hass.io deals with it. Speaking of HA - I was inspired by this blog to dockerize things: https://www.smarthomebeginner.com/insta ... ker-linux/.

Yes, I do fear Kubernetes adds unnecessary complexity right now - maybe it would be worth it in a more distributed setup with multiple Odroids.

Exposing multiple services on port 80 - well, I only have one right now, wordpress, and for other things I can just redirect to other ports. On the XU4 however I run must of the stuff with network_mode: host to remove the overhead of the virtual networking, plus they all play nice as they use wildly different ports(radarr, sonarr, emby etc). Now, it would be entirely possible to spin up a reverse proxy service if you needed to have multiple things on port 80/443 but for now it works, this is what this guy did: https://www.smarthomebeginner.com/traef ... or-docker/. I've used nginx for that in the past and also HAProxy(horrible user experience).

RAM usage - this is where it gets tricky indeed, now I am not sure I can measure it since going with docker has allowed me to diversify the setup more than what I had originally (oh, that reminds, whenever there was a mono update, radarr, sonarr and sometimes emby would break and I had to manually intervene and fix things). BUT, I did feel the need for a swapfile as the RAM would get exhausted at times. That being said, performance is fine for now(except for when the disk gets corrupted every night, ha ha, this is the story from the other thread).

Bottom, bottom line, if I need to ditch the cloudshell for another OS/board etc, all I have to do is plug in the drives(pvscan, mount the volumes etc) and then docker-compose up everything. And also a diversified, appliance style setup.

Hope my rant makes sense !

[memeka]

I know what you mean. I also started to dockerize a lot because "reinstalling" is now just a "docker-compose up"
I see we have some similar software and sources here and there

Code: Select all

# docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                  PORTS                                                                                                        NAMES
11dd8119c3e9        jupyter/scipy-notebook       "tini -g -- start-no…"   11 days ago         Up 8 days               0.0.0.0:8888->8888/tcp                                                                                       jupyter
64fdb59e056f        pihole/pihole:latest         "/s6-init"               2 months ago        Up 2 months (healthy)   0.0.0.0:53->53/tcp, 0.0.0.0:53->53/udp, 0.0.0.0:32775->67/udp, 0.0.0.0:9001->80/tcp, 0.0.0.0:9002->443/tcp   pihole
34f66a60cda7        portainer/portainer          "/portainer"             9 months ago        Up 2 months             0.0.0.0:9000->9000/tcp                                                                                       portainer
60fcd1fe6635        linuxserver/jackett:latest   "/init"                  18 months ago       Up 2 months             0.0.0.0:9117->9117/tcp                                                                                       jackett
cd193a2a47a5        linuxserver/sonarr:latest    "/init"                  18 months ago       Up 2 months             0.0.0.0:8989->8989/tcp, 0.0.0.0:9898->9898/tcp                                                               sonarr

and

Code: Select all

$ docker ps
CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                                      NAMES
a005c0a424d5        celery                         "supervisord -c /app…"   50 minutes ago      Up 50 minutes       0.0.0.0:5555->5555/tcp, 80/tcp, 0.0.0.0:10000->10000/tcp   celery
e741590b552e        grafana/grafana:6.5.1-ubuntu   "/run.sh"                6 hours ago         Up 6 hours          3000/tcp, 0.0.0.0:9000->9000/tcp                           grafana
ff3079848241        telegraf:1.12                  "/entrypoint.sh tele…"   6 hours ago         Up 6 hours          8092/udp, 8125/udp, 8094/tcp                               telegraf
9418fe27b002        influxdb:1.7                   "/entrypoint.sh infl…"   6 hours ago         Up 6 hours          0.0.0.0:8086->8086/tcp                                     influxdb
c35991054fd6        rabbitmq:3.7-management        "docker-entrypoint.s…"   6 hours ago         Up 6 hours          4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp        rabbitmq
e449d6eab71b        oznu/homebridge:debian         "/init"                  2 weeks ago         Up 3 days                                                                      homebridge

[mad_ady]

Thanks for the traefik tip! I need to do some reading.
So, you're recommending using docker-compose instead of archaic docker run, right?

I need to get my method right so it doesn't become a burden...

[memeka]

i use docker-compose for the apps that need to run together.... e.g. for my solar monitoring i need to run influxdb, telegraf, grafana, rabbitmq and celery (for scheduled tasks), so i have a docker-compose file to set them all up. homebridge doesn't need anything else to run, so i used docker run

[mad_ady]

How do you remember the parameters for docker run? You can't easily derive them from a running container (e.g. when you pass environment variables or special parameters). Do you rely only on bash history?

[memeka]

How do you remember the parameters for docker run? You can't easily derive them from a running container (e.g. when you pass environment variables or special parameters). Do you rely only on bash history?

bash history )
but indeed you can make a docker-compose just for having the parameters in the yml

[joshua.yang]

I also strongly recommend using docker-compose rather than archaic docker run by separated commands.
docker-compose not only compromises among the docker images but also gives you chances to manage the services, acting at once.
Besides, it is very easy.