Anyone else just sick of trying to follow guides that cover 95% of the process, or maybe slightly miss a step and then spend hours troubleshooting setups just to get it to work?
I think I just have too much going in my “lab” the point that when something breaks (and my wife and/or kids complain) it’s more of a hassle to try and remember how to fix or troubleshoot stuff. I lightly document myself cuz I feel like I can remember well enough. But then it’s a style to find the time to fix, or stuff is tested and 80%completed but never fully used because life is busy and I don’t have loads of free time to pour into this stuff anymore. I hate giving all that data to big tech, but I also hate trying to manage 15 different containers or VMs, or other services. Some stuff is fine/easy or requires little effort, but others just don’t seem worth it.
I miss GUIs with stuff where I could fumble through settings to fix it as is easier for me to look through all that vs read a bunch of commands.
Idk, do you get lab burnout? Maybe cuz I do IT for work too it just feels like it’s never ending…
I deliberately have not used docker at home to avoid complications. Almost every program is in a debian/apt repo, and I only install frontends that run on LAMP. I think I only have 2 or 3 apps that require manual maintenance (apart from running “apt upgrade”). NextCloud is 90% of the butthurt.
I’m starting to turn off services on IPv4 to reduce the network maintenance overhead.
I wouldn’t say im stick of it, but it can be a lot of work. It can be frustrating at times, but also rewarding. Sometimes I have to stop working on it for a while when I get stuck.
In any case, I like it a lot better than being Google’s bitch.
I have to stop working on it for a while when I get stuck.
I feel you there bro. Sometimes, when I’m creating a piece of music, I get to a point where, I’m just not making any progress, I’ll step of for a piece, let it simmer for a bit. Same with servers in general for me. It’s the reason I have a test server and have, in the past, leaned a bit heavily on a few backups. LOL! I can screw something up quick when I’m frustrated. The reward for me is learning something new. It’s a rewarding and useful hobby for me. among others.
I do it in sprints. I’ll set up a service, test it, get it working, then share it with the family.
I hear you on the instructions. A lot of these are pet projects that just happen to work well enough to share, so a bit of work is needed to implement them. If you document for others, you find that you can’t ever put every step in there because you can’t control all the variables.
Just 15 containers? lol
do you get lab burnout
Not really. I have everything set the way I want it and it’s stable. On occasion, I’ll see a container that catches my fancy, so I’ll spin it up on a test server, dick around with it, and monitor it before I ever decide to put it on my production server. On occasion I’ll have to fix, or adjust something. Most of the time I’m just enjoying it. I wouldn’t say I was running anything super complex tho.
As far as time, I’ve got you beat there most likely. Used to be lickity-split, but then you get old, things slow down. LOL Also, there is only one user…me. I realize you have family, but my hard and fast rule is: Multiple users cause issues, so I don’t share. I’d say, go spend your time with the family. That’s the most important.
I’m with you on the incomplete guides. There always seem to be that one ‘secret’ ingredient’ that just didn’t get documented. And to the devs of the opensource software, me love you long time, but please include a screenshot.
Proxmox?
And yes. Its like a full time job to homelab. Or a part time job. Its just hard, and sometimes things just don’t work.
I guess one answer is to pick your battles. You can’t win them all. But things are objectively better than they were in the past.
It can absolutely be overwhelming, and very easy to forget specifics over a long time. It’s partly why I don’t really go for CLI apps, and ~all of my apps are just Ansible manifests. Which apps are causing the biggest problems for your family?
What exactly is breaking each of these times? Guides that cover 95% sound pretty solid to me. It’s hard to write a guide covering 100% of scenarios. Admittedly I also worked in the field, but the field is extremely wide so maybe there’s some knowledge areas to deepen that are commonly giving you problems and/or move towards a less brittle setup.
Re-evaluating what’s important is important. If it’s not fun then you should reflect on having the right balance of what is helping you and your family vs causing excessive stress. IMO the “avoid all tech companies” is slightly overblown (blasphemous, I know). It’s a good guiding principle but it’s fine to “buy services” that make your life better. For example, I self host a lot, but I was totally fine buying a finances tracking app (the spreadsheet-based one) because it’s doing a lot of heavy lifting that I can’t reasonably do myself at the level of convenience I want.
Well, I’ll share an example. Choosing between Traccar and OwnTracks. I’ve run a lot of stuff on Raspberry Pis, I like it, but, do I keep setting up new devices just to continue more devices or do I dump some for a Linux desktop and move a lot to containers? But that’s more work lol. Aren’t there different versions of docker, too? I recall fucking a service up one time using the wrong documentation once.
I think part of my problem is I’ve pieced stuff together slowly and it feels like a fragile balance, but at work I’ve got more access to resources… And budget lol
I paid around 200€ for a used HP OEM desktop machine. It got a i7 9700 and 32 GB of RAM in it. Still idles at a pretty low power consumption. But I never have to worry about resources, haha. I come from a long history of windows, just recently switched one of my main PCs over to Linux. I like a GUI, still. I got unraid for my server, back when the lifetime licences were still the norm. Makes it really easy to manage services, especially in conjunction with storage (say Immich or Navidrome). Containers and VMs are managed via a GUI and super easy to set up. I work IT-adjacent, but I’m far from being as professional as probably most people here, so this works fine for me.
I’m sick of everything moving to a docker image myself. I understand on a standard setup the isolation is nice, but I use Proxmox and would love to be able to actually use its isolation capabilities. The environment is already suited for the program. Just give me a standard installer for the love of tech.
unless you have zillion gigabytes of RAM, you really don’t want to spin up a VM for each thing you host. the separate OS-es have a huge memory overhead, with all the running services, cache memory, etc. the memory usage of most services can largely vary, so if you could just assign 200 MB RAM to each VM that would be moderate, but you can’t, because when it will need more RAM than that, it will crash, possibly leaving operations in half and leading to corruption. and to assign 2 GB RAM to every VM is waste.
I use proxmox too, but I only have a few VMs, mostly based on how critical a service is.
For VMs, I fully agree with you, but the best part about Proxmox is the ability to use containers, or CTs, which share system resources. So unlike a VM, if you specify a container has two gigs of RAM, that just means that it has two gigs of RAM that it can use, unlike the VM where it’s going to use that amount (and will crash if it can’t get that amount)
These CT’s do the equivalent of what docker does, which is share the system space with other services with isolation, While giving an easy to administrate and backup system, while keeping it able to be seperate by service.
For example, with a Proxmox CT, I can do snapshots of the container itself before I do any type of work, if where if I was using Docker on a primary machine, I would need to back up the Docker container completely. Additionally, having them as CTs mean that I can run straight on the container itself instead of having to edit a Docker file which by design is meant to be ephemeral. If I had to take troubleshooting bare bones versus troubleshooting a Docker container, I’m going to choose bare bones every step of the way.(You can even run an Alpine CT if you would rather keep the average Docker container setup)
Also for the over committing thing, be aware that your issue you’ve stated there will happen with a Docker setup as well. Docker doesn’t care about the amount of RAM the system is allotted. And when you over-allocate the system, RAM-wise, it will start killing containers potentially leaving them in the same state.
Anyway, long story short, Docker containers do basically the same thing that a Proxmox CT does. it’s just ephemeral instead of persistent, And designed to be plug-and-go, which I’ve found in the case of running a Proxmox-style setup, isn’t super handy due to the fact that a lot of times I would want to share resources such as having a dedicated database or caching system, Which is generally a pain in the butt to try to implement on Docker setups.
I’m really confused here, you don’t like how everything is containerized, and your preferred method is to run Proxmox and containerize everything?
I don’t like how everything is docker containerized.
I already run proxmox, which containerizes things by design with their CT’s and VM’s
Running a docker image ontop of that is just wasting system resources. It doesn’t make sense to run a CT or VM for a container, just to put docker on it and run another container via that. It also completly bypasses everything that proxmox provides you for snapshotting and backup because proxmox’s system is for the entire container, and if all services are running on the same container all services are going to be snapshotted.
My current system allows me to have per service snapshots(and backups), all within the proxmox webUI, all containerized, and all restricted to their own resources. Docker is just not needed at this point.
A docker system just adds extra headway that isn’t needed. So yes, just give me a standard installer.
Nothing is “docker containerized”. Docker is just a daemon and set of tools for managing OCI compliant containers.
Running a docker image ontop of that is just wasting system resources.
No? If you spun up one VM in Proxmox and installed docker and used it to run 10 containers, that would use fewer system resources than running 10 LXC containers directly on Proxmox.
Like… you don’t like that the industry has adapted this efficient, portable, interchangeable, flexible, lightweight, mature technology, because you prefer the one that is heavier, less flexible, less portable, non-OCI compliant alternative?
are you are saying running docker in a container setup(which at this point would be 2 layers deep) uses less resources than 10 single layer deep containers?
I can agree with the statement that a single VM running docker with 10 containers uses less than 10 CT’s with docker installed then running their own containers(but that’s not what I do, or what I am asking for).
I currently do use one CT that has docker installed with all my docker images, which I wouldn’t do if I had the ability not to but some apps require docker) but this removes most of the benefits you get using proxmox in the first place.
One of the biggest advantages of using the hypervisor as a whole is the ability to isolate and run services as their own containers, without the need of actually entering the machine. (like for example if I"m screwing with a server, I can just snapshot the current setup and then rollback if it isn’t good) Throwing everything into a VM with docker bypasses that while adding headway to the system. I would need to backup the compose file (or however you are composing it) and the container, and then do my changes. My current system is a 1 click make my changes, if bad one click to revert.
For resource explanation. Installing docker into a VM on proxmox then running every container in that does waste resources. You have the resources that docker requires to function (which is currently 4 gigs of ram per their website but when testing I’ve seen as low as 1 gig work fine)+ cpu and whatever storage it takes up which is about half a gig or so) in a VM(which also uses more processing and ram than CT’s do as they no longer share resources). When compared to 10 CT’s that are finetuned to their specific app, you will have better performance running the CT’s than a VM running everything, while keeping your ability to snapshot and removing the extra layer and ephemeral design that docker has(this can be a good and bad thing, but when troubleshooting I learn towards good).
edit: clarification and general visibility so it wasnt bunched together.
are you are saying running docker in a container setup(which at this point would be 2 layers deep) uses less resources than 10 single layer deep containers?
If those 10 single layer deep containers are Proxmox’s LXC containers then yes, absolutely. OCI containers are isolated processes that run single services, usually just a single binary. There’s no OS, no init system. They’re very lightweight with very little overhead. They’re “containerized services”. LXC containers on the other hand are very heavy “system containers” that have a full OS and user space, init system, file systems etc. They are one step removed from being full size VMs, short of the fact that they can share the hosts kernel and don’t need to virtualize. In short, your single LXC running docker and a bunch of containers inside of it is far more resource efficient than running a bunch of separate LXC containers.
One of the biggest advantages of using the hypervisor as a whole is the ability to isolate and run services as their own containers, without the need of actually entering the machine
I mean that’s exactly what docker containers do but more efficiently.
I can just snapshot the current setup and then rollback if it isn’t good
I mean that’s sort of the entire idea behind docker containers as well. It can even be automated for zero downtime updates and deployments, as well as rollbacks.
When compared to 10 CT’s that are finetuned to their specific app, you will have better performance running the CT’s than a VM running everything
That is incorrect. Let’s break away from containers and VMs for a second and look deeper into what is happening under the hood here.
Option A (Docker + containers): One OS, One Init system, one full set of Linux libraries.
Option B (10 LXC containers): Ten operating systems, ten separate init systems, 10 separate sets of full Linux libraries.
Option A is far more lightweight, and becomes a more attractive option the more services you add.
And not only that, but as you found out, you don’t need to run a full VM for your docker host. You could just use an LXC. Though in that case I’d still prefer the one VM, so that your containers aren’t sharing your Proxmox Host’s kernel.
oh, LXC containers! I see. I never used them because I find LXC setup more complicated, once tried to use a turnkey samba container but couldn’t even figure out where to add the container image to LXC, or how to start if not that way.
but also, I like that this way my random containerized services use a different kernel, not the main proxmox kernel, for isolation.
Additionally, having them as CTs mean that I can run straight on the container itself instead of having to edit a Docker file which by design is meant to be ephemeral.
I don’t understand this point. on docker, it’s rare that you need to touch the Dockerfile (which contains the container image build instructions). did you mean the docker compose file? or a script file that contains a docker run command?
also, you can run commands or open a shell in any container with docker, except if the container image does not contain any shell binary (but even then, copying a busybox or something to a volume of the container would help), but that’s rare too.
you do it like this: docker exec -it containername command. bit lengthy, but bash aliases helpAlso for the over committing thing, be aware that your issue you’ve stated there will happen with a Docker setup as well. Docker doesn’t care about the amount of RAM the system is allotted. And when you over-allocate the system, RAM-wise, it will start killing containers potentially leaving them in the same state.
in docker I don’t allocate memory, and it’s not common to do so. it shares the system memory with all containers. docker has a rudimentary resource limit thingy, but what’s better is you can assign containers to a cgroup, and define resource limits or reservations that way. I manage cgroups with systemd “.slice” units, and it’s easier than it sounds
NixOS for the win! Define your system and services, run a single command, get a reproducible, Proxmox-compatible VM out of it. Nixpkgs has basically every service you’d ever want to selfhost.
You can still use VMs and do containers in there. That’s what I do, makes separating different services very easy.
This is what I currently do with non-specialized services that require Docker. I have one container, which runs Docker Engine, and I throw everything on there, and then if I have a specialized container that needs Docker, I will still run its own CT. But then I use Docker Agent, So I can use one administration panel.
It’s just annoying because I would rather just remove Docker from the situation because when you’re running Proxmox, you’re essentially running a virtualized system in a virtualized system because you have Proxmox, which is the bare bones running a virtualized environment for the container, which is then running a virtualized environment for the Docker container.
running a virtualized environment for the container, which is then running a virtualized environment for the Docker container.
Neither Linux containers nor Docker containers are virtualized.
I think we might have a different definition of Virtualized and containers. I use IBM’s and Comptias definitions.
IBM’s definition is
Virtualization is a technology that enables the creation of virtual environments from a single physical machine, allowing for more efficient use of resources by distributing them across computing environments.The IBM page themselves acknowledges that containers are virtualization on their Containers vs Virtual Machines page. I call virtualization as an abstraction layer between the hardware and the system being run.
Comptia’s definition of containers would be valid as well. Which states that containers are a virtualization layer that operates at the OS level and isolates the OS from the file system. Whereas virtual machines are an abstraction layer between the hardware and the OS.
I grew this terminology from my comptia networking+ book from 12 years ago though, which classifies Virtualization as “a process that adds a layer of abstraction between hardware and the system” which is a dated term since OS level virtualization such as Containers wasn’t really a thing then.
I thought that was the point of supporting OCI in the latest version so you can pull docker images and run them like an lxc container
If there’s a way of pulling a Docker container and running it directly as a CT on Proxmox, please fill me in. I’ve been using it for a year and a half to two years now, but I haven’t seen any ability to directly use a Docker container as an LXC.
This was added in Proxmox 9.1
I reject a lot of apps that require a docker compose that contains a database and caching infrastructure etc. All I need is the process and they ought to use SQLite by default because my needs are not going to exceed its capabilities. A lot of these self hosted apps are being overbuilt and coming without defaults or poor defaults and causing a lot of extra work to deploy them.
Some apps really go overboard, I tried out a bookmark collection app called Linkwarden some time ago and it needed 3 docker containers and 800MB RAM
Found an alternative solution to recommend?
No, but I’d like to hear it if anyone else finds one
Databases.
I ran PaperlessNGX for a while, everything is fine. Suddenly I realize its version of Postgresql is not supported anymore so the container won’t start.
Following some guides, trying to log into the container by itself, and then use a bunch of commands to attempt to migrate said database have not really worked.
This is one of those things that feels like a HUGE gotcha to somebody that doesn’t work with databases.
So the container’s kinda just sitting there, disabled. I’m considering just starting it all fresh with the same data volume and redoing all that information, or giving this thing another go…
…But yeah I’ve kinda learned to hate things that rely on database containers that can’t update themselves or have automated migration scripts.
I’m glad I didn’t rely on that service TOO much.
Its a big problem. I also dump projects that don’t automatically migrate their own SQLite scehema’s requiring manual intervention. That is a terrible way to treat the customer, just update the file. Separate databases always run into versioning issues at some point and require manual intervention and data migration and its a massive waste of the users time.
Yeah that’s part of having a hobby. If you do it for work too I can understand getting sick of it. But, no one is making you do it. If you don’t enjoy it, don’t do it.
While this might be a healthy outlook, these days more and more people do not feel like self hosting is a hobby or an option, but a necessity for a free and fair society.
This sooo much. I’m not a tech person but I’m trying to learn because the giant corporations are clearly evil. I just want to have a modicum of privacy in my corner of the world so here I am trying to figure out how to self host some basic services.
This. I self host some things because it’s just fun, other things because of censorship, other things because of privacy. I probably wouldn’t have Nextcloud if Google wasn’t collecting so much data. Probably wouldn’t be self-hosting my blog if content weren’t as censored everywhere. I probably would still be self-hosting a Minecraft server with a small website for said server that the members of the server can contribute to when they find/do something cool.
Nextcloud is on my list lol, but I need to run a separate box for it I think vs visualizing. It would be easier/cleaner and more reliable.
Yeah, it’s definitely one of those that’s also just… Useful. I usually don’t go for software that’s trying to do too much, but for some reason I don’t mind having nextcloud as 10 different things xD Sync files, sync podcast listens, sync my RSS feeds… A lot of things all in one
Use portainer for managing docker containers. I prefer a GUI as well and portainer makes the whole process much more comfortable for me.
+1 for Portainer. There are other such options, maybe even better, but I can drive the Portainer bus.
just know that sometimes their buggy frontend loads the analytics code even if you have opted outm there’s an ages old issue of this on their github repo, closed because they don’t care.
It’s matomo analytics, so not as bad as some big tech, but still.
Why did I never think of that?! That would make sense lol. Thank you!
No problem. I have been using it for a while and I really like it. There’s nothing stopping you from doing it the old fashioned way if you find you don’t like portainer but once you familiarize yourself with it I think you’ll be hooked on the concept.
If you’ll let me self promote for a second, this was part of the inspiration for my Ansible Homelab Orchestration project. After dealing with a lot of those projects that practically force you to read through the code to get a working environment, I wanted a way to reproducably spin up my entire homelab should I need to move computers or if my computer dies (both of which have happened, and having a setup like this helped tremendously). So far the ansible playbook supports 117 applications, most of which can be enabled with a single configuration line:
immich_enabled: true nextcloud_enabled: trueAnd it will orchestrate all the containers, networks, directories, etc for you with reasonable defaults. All of which can be overwritten, for example to enable extra features like hardware acceleration:
immich_hardware_acceleration: "-cuda"Or to automatically get a letsencrypt cert and expose the application on a subdomain to the outside world:
immich_available_externally: trueIt also comes with scripts and tests to help add your own applications and ensure they work properly
I also spent a lot of time writing the documentation so no one else had to suffer through some of the more complicated applications haha (link)
Edit: I am personally running 74 containers through this setup, complete with backups, automatic ssl cert renewal, and monitoring
Yeah, self promote away lol
I hesitate to bring this up because you’ve clearly already done most of the hard work, but I’m planning on attending the following conference talk this weekend that might be of interest to you: https://fosdem.org/2026/schedule/event/VEQTLH-infrastructure-as-python/
No that’s totally fair! I’m a huge fan of making things reproducible since I’ve ran into too many situations where things need to be rebuilt, and always open to ways to improve it. At home I use ansible to configure everything, and at work we use ansible and declare our entire Jenkins instance as (real) code. I don’t really have the time for (and I’m low-key scared of the rabbit hole that is) Nix, and to me my homelab is something that is configured (idempotently) rather than something I wanted to handle with scripts.
I even wrote some
pytest-like scripts to test the playbooks to give more productive errors than their example errors, since I too know that pain well :DThat said, I’ve never heard of PyInfra, and am definitely interested in learning more and checking out that talk.
Do you know if the talk will be recorded? I’m not sure I can watch it live.Edit: Found a page of all the recordings of that room from last year’s event https://video.fosdem.org/2025/ua2220/ So I’m guessing it will be available. Thank you for sharing this! :DI love the “Warning: This talk may cause uncontrollable urges to refactor all your Ansible playbooks” lol I’m ready
@Dylancyclone @selfhosted This looks very useful. I will study your docs and see if it’s right for me. Thanks for sharing!
That’s neat. I never gave ansible playbooks any thought because I thought it would just add a layer of abstraction and that containers couldn’t be easier but reading your post I think I have been wrong.
While it is true that Ansible is a different tool that you need to learn the basics of (if you want to edit/add applications), all of the docker stuff is pretty comparable. For example, this is the equivalent of a docker compose file for SilverBullet (note taking app): https://github.com/Dylancyclone/ansible-homelab-orchestration/blob/main/roles/silverbullet/tasks/main.yml
You can see it’s volumes, environment variables, ports, labels, etc just like a regular docker compose (just in a slightly different format, like environment variables are listed as
envinstead ofenvironment), but the most important thing is that everything is filled in with variables. So for SilverBullet, any of these variables can be overwritten, and you’d never have to look at/tweak the “docker compose.” Then, if any issue is found in the playbook, anyone can pull in the changes and get the fix without any work from themselves, and if manual intervention is needed (like an app updated and now requires a new key or something), the playbook can let you know to avoid breaking something: https://dylancyclone.github.io/ansible-homelab-orchestration/guides/updating/#handling-breaking-changes
I’m currently running three hosts with a collection of around 40 containers.
one is the house host, one is the devops host, and one is the AI host.
I maintain images on the devops host and deploy them regularly. when one goes down or a container goes down, I am notified through mqtt on my phone. all hosts, services, ports, certs, etc are monitored.
no problems here. git gud I suppose?
And honestly, 40 isn’t even impressive. I run more than that on one host. Containers make life so much easier is unreal.
Once you understand them, I suppose its easier. I’ve got a mix of win10, Linux VMs, RPis, and docker.
Having grown up on Windows, it’s second nature now and I do it for work too. I stated on Linux only around 2010 or so but kept flipping between the2 . anymore, trying to cut the power bill and went RPi but also trying to cut others and so docker is still relatively new in the last few years. Understand that I also do it few and far between at times on projects so is hard to dedicate time to learn enough to be comfortable. It also didn’t help I started on Docker Desktop and apparently everyone hates that and may have been a part of my problem adopting it.
I probably also started with linux seriously around that time frame. I was also a Windows admin back then. Transitioning to Linux and containers was the best thing ever. You get out of dependency hell and having kruft all over your filesystem. I’m extremely biased though, I work for Red Hat now. Containers and Linux are my day job.
Dang, how’d you make that transition? Are you a dev or SWE?
I just liked linux better so I learned it. That’s kind of my whole career, I want to do something so I get certified in it and start looking to get into it. I’m in consulting. I come in and help people setup OpenShift while teaching them how to use it and then move on to the next customer.
You’re not alone.
The industry itself has become pointlessly layered like some origami hell. As a former OS security guy I can say it’s not in a good state with all the supply-chain risks.
At the same time, many ‘help’ articles are karma-farming ‘splogs’ of low quality and/or just slop that they’re not really useful. When something’s missing, it feels to our imposter syndrome like it’s a skills issue.
Simplify your life. Ditch and avoid anything with containers or bizarre architectures that feels too ontricate. Decide what you need and run those on really reliable options. Auto patching is your friend (but choose a distro and package format where it’s atomic and rolls back easily).
You don’t need to come home only to work. This is supposed to be FUN for some of us. Don’t chase the Joneses, but just do what you want.
Once you’ve simplified, get in the habit of going outside. You’ll feel a lot better about it.
That’s true, I’ve done a lot of stuff as testing that I thought would be useful services but then never really got used by me, so I didn’t maintain.
I didn’t take the time to really dive in and learn Docker outside of a few guides, probably why is a struggle…
Yes, I get lab burnout. I do not want to be fiddling with stuff after my day job. You should give yourself a break and do something else after hours, my dude.
BUT
I do not miss GUIs. Containers are a massive win in terms because they are declarative, reproducible, and can be version controlled.
Yeah, since Christmas, I more it sounds silly, but I’ve been playing a ton of video games with my kids lol. But not like CoD, more like Grounded 2, Gang Beasts, and Stumble Guys lmao
You’re doing i right. Playing cool games with your kids sounds like a blast and some great memories :)
I definitely feel the lab burnout, but I feel like Docker is kind of the solution for me… I know how docker works, its pretty much set and forget, and ideally its totally reproducible. Docker Compose files are pretty much self-documenting.
Random GUI apps end up being waaaay harder to maintain because I have to remember “how do I get to the settings? How did I have this configured? What port was this even on? How do I back up these settings?” Rather than a couple text config files in a git repo. It’s also much easier to revert to a working version if I try to update a docker container and fail or get tired of trying to fix it.
