Personally I’m starting to get into the realm of a mini data center. I’m lucky enough to live in a place that I have access to FiOS business class at a good price.
I’m just enjoying having access to things and never really knowing if google is down or some other crap is going on. I have a lot of tweaking to get things more autonomous, but otherwise, it is very fun. The best part is that other companies have much less data on me.
As for hardware, I’m running:
Custom
HP Z400
HP Mini Tower
Power
Networking
Xeon E3-1220 v5 @ 3.00GHz
Xeon W3520 @ 2.67GHz
Intel i3-6100T @ 3.20GHz
2x 1500VA battery backups
Ubiquiti Edgemax 4
16GB Ram
24GB Ram
8GB Ram
Auto switching dual input PDU
24 port edgeswitch
8 Hot Swap HDD bays (4x 1TB WD-Greens)
2x 1TB WD-Blacks
2TB WD-Red
About 45 minutes battery time
I’ve been really getting sucked into the hosted services, too:
I have some cloud VMs that I use for my more public facing services, but also a home server system. I have 2 almost identical servers in my house running Ubuntu Impish server edition and using CRM to balance services and VMs between them. This way I can take a machine down and not lose any functionality in my house. I also have a little Asus puck style machine that runs my TV/Stereo system and provides the tie breaking vote in the above CRM configuration. I’m using Zwave and Openhab on one of the VMs to fully automate my lights, locks, heating, and sensors. I have Mycroft running on another VM to provide an easier interface to Openhab, along with DeepSpeech for local speech to text (screw google), and MaryTTS to provide text to speech. While I have been using MD RAID with DRBD to keep the machines in sync, and GFS2 on top so that they both have access to the data, I am currently half way through switching over to a ZRAID system (GFS2 is very slow). I also have the obligatory laptops and pies running, including a pi3 with a cluster hat of pi0s on it, but they are just for hobby fun right now.
That is nice. I still haven’t setup a high availability system yet. I’m really intrigued on what you did to get Mycroft off of the web. I want to use him, but I don’t like relying on a central server.
What is the core that is running your Zwave stuff? I’m relying on zwave-js and I can’t determine if there is a network bottleneck or if zwave-js is just not running as snappy as it should. I don’t get much lag, but there is some there, but only some of the time.
It’s basically just a python server app. The you just have to adjust your local mycroft.conf like this:
“stt”: {
“deepspeech_server”: {
“uri”: “http://127.0.0.1:8080/stt”
},
“module”: “deepspeech_server”
},
For MaryTTS, you can find it here: http://mary.dfki.de/
It’s just a Java server app. And you just add this to your local mycroft.conf:
As for Zwave, I have the Aeotec Z-Stick USB stick. It originally worked very well but has slowed considerably as my system grew. I have 2 doors, a thermostat, 8 lights, and 10 sensors on it. I think I would probably not use Zwave if I started over again. I get enough lag that if I walk through my house and trip 4 or 5 sensors in a row, it can take a minute or 2 for the system to respond to the last sensor. This does not happen with the devices that are on other buses in the system.
Hearing your problems with Z-wave, got me thinking. I found out that there is a hardware max of 232 devices. I’m nowhere near that and with you having the same issues, I went looking. It turns out (so far with my testing) that the issue was the devices sending power usage data. I had 6 plugs that were sending 30 second updates & updates on changes with the power usage.
I don’t need that and disabled it. Lo and behold, my switches are reacting much fast and the others that were being a pain, aren’t. We shall see what the future holds, but that may be your issue, too.
I looked at my network usage here too when I was troubleshooting this originally. I have my devices set to only transmit on update, which really translate to me using a door or light. My sensors only transmit updates if they see/don’t see motion (they have a 4 minute timeout between those states), and a general update once an hour.
I have a feeling that my issues stem from a Zwave driver update/Openhab update. My system ran flawlessly for a few years, then one of my Zwave switches died. The replacement switch had an updated firmware, which required an updated driver, which then required an updated Openhab. The new Openhab brings a GUI that is much more user friendly than the old text file config system, but my Zwave network has not worked the same since. I often think about trying HomeAssistant, but I have a lot of custom rules on my current system. Maybe over Christmas I might make a test VM and see how just my Zwave stuff reacts on it.
I’d love to host stuff at home and I‘d love to to an episode or series of episodes on it, but I really don’t see that working for me or anyone else living in a similar situation. I mean, bandwidth isn’t generally the issue in big cities in Germany anymore. My fibre at home could probably do it. But German ISPs are a fucking pain in the ass, even with a business account. Sure. They’d give me a static IP but that’s about it. I looked into it when I went to start my own business and it’s a hassle. They still block all kinds of shit and if you start sending email, they get really pissed.
I would research it more but my real issue is the hardware. I live in a relatively small flat and I just have nowhere to put the servers. I could put them in the office, but then I can’t podcast anymore because of the fan noise. And even then the wife would probably kill me.
I have a friend who runs servers in a similar flat in a storage cupboard and when Katy saw that she was like: we will never do this. Just so you know.
Still. Great thread! I will be watching it with interest.
I actually crossed an offhanded comment by someone, that I hadn’t thought about, until they mentioned it. It was about how Laptops are some of the best small servers ever. They are small, quite, and come with their own battery back-ups.
I got into it because I crossed a great deal with a client that had old enterprise server hardware that needed “recycling.” I just had to deal with the very loud fans on the units. Later on I just upgraded.
But the laptops with built-in graphics are really nice for a home Jellyfin media server with hardware transcoding abilities! Perfect for dipping the toes.
That is true. I’ve repurposed old laptops like that for years. Needs to be a proper computer, though. All of these modern tablet-slidy-thingies don’t have Ethernet ports anymore!
I was running my email/web/svn server from my house for years. On Gentoo. The problem wasn’t actually Gentoo, it was that only one provider where I am had static IP addresses and let me host my own servers, and its a wireless connection. On a good day I get 5Mbps, on a bad day I wish I had dialup still. So I moved everything to a VM provider and my house connects to it via a VPN to keep external access to my stuff that can’t be moved from the house.
I could actually get a fixed IP on a business line here. But I have no idea how the provider would deal with it should that get blacklisted by anti-spam services. They very often blacklist whole IP ranges.
Not that I ever want to run my own email server. I decided that in about 2006 or so. It’s a fucking nightmare and a lot of hassle I don’t need.
The only thing I self-host is a private git server (gitolite-powered) on my Raspberry Pi in a cupboard. It also acts as NAS, media server, backup server, fax server (yep, an actual modem and a phone line; yep, in 2021; yep, medicine is conservative, and I mean it), and a 24/7 Syncthing node but I don’t consider those applications as “self-hosting”.
I toyed with the idea of self-hosting more things (such as my website, my Debian repository, my FreshRSS instance, my CalDAV/CardDAV server, etc, all the stuff I currently use a VPS for, but I’ve tabled it for now, for two reasons. First, the apartment building I live in was built in 1968, and while the electrical wiring in the apartment itself is OK (I own the apartment, and I had all the wiring removed and done from scratch when I first bought it), the wiring in the rest of the building is old and was never meant for the modern loads, so we have some power issues every now and then. Second, I only have one ISP cable in my apartment, and I don’t want to drill holes and lay a redundancy one now (though I’ll consider it when we’ll have a renovation next time).
I have been sharing this sentiment for ages, but I am considering the idea more and more now. Not self-hosting (at least no with my current living arrangements, see above), but running my own mailserver on a VPS, I am really considering it.
Thing is, every couple of years I get so fed up with one of my mail hoster’s idiosyncrasies that I switch a mail hoster. As a result, I’ve used quite a number of those, and I have yet to find the one to satisfy me 100%. More and more I’m thinking I should just go and do everything to my liking myself. Of course, the admin tax on a mailserver is probably higher than for any other application, and I’ll need to improve my skills a lot it I ever consider going there, but it’s definitely doable.
I’m not so sure. It’s gotten a lot harder recently. I’m talking to the company that hosts my mail services once in a while for stories and the amount of shit they go through is unbelievable.
It takes a lot of effort to build trust with other email providers. It’s almost guaranteed these days that, if you just set up your own server on an IP the anti-spam services don’t know about, you will be treated as a spammer. If your server sends emails to services like GMail, your mail will end up in the spam folder of the recipient automatically. This even happens with my provider sometimes and they are quite big and have been around for ages. Some servers even delete emails from servers that are not in some kind of trusted list outright.
The spam and phishing situation in the last few years has caused a lot of complications in this regard. If anything, running your own mail server has gotten a lot harder over the last decade.
I actually think mail servers have gotten easier, or I have just become desensitized to it. In the old days, it was ridiculously easy to have a couple of false spam reports get your IP on one of the blacklists, and crazy hard to get that reversed. Some of the more modern technologies that have been added, like SPF, DMARC, and DKIM have really cut back on that. Even spam prevention has gotten easier with tricks like automatic graylisting. As I write this, the obvious downside of making the initial setup and learning curve steeper become apparent to me, since I only had to adapt and integrate over the years, but it is still nice for me that I can set everything the way I want it and my integrated systems can use it for notifications and other various automations.
You’re right. Some servers are outright paranoid. There’s a university in Moscow that I haven’t been able to email from my personal address with the last four of my mail providers, including the current one; they seem to only be accepting Gmail and Mail.Ru (maybe also Yandex and Yahoo, but I’m not sure) emails, and their server just silently drops the messages it doesn’t like. I already got used to tell their employees and students to use their personal mail instead if they want to hear back from me.
I think it’s a big problem, actually, and I think it paves the road for the one truly federated system that actually works still to turn into a closed ecosystem of a few big providers. That’s bad. And I’m happy I can afford urging people to tell their company IT that their mail server incoming filters need to be fixed (and several companies’ servers have indeed been fixed this way). But that’s me; people I write to usually want my email enough to tell their sysadmins to stop impeding the company’s work by their paranoia.
I mean, true. But it is what a hell of a lot of mail systems use. Anyway, I’m very happy not to administer any mail systems at all. Can’t do everything, I say. I’m trying so many new things constantly already, there’s just some stuff that I’ve recognised just isn’t for me though.
I’ve used a few different mail hosting options, and MailCow is one of the best, in my opinion. The initial DNS setup is long (like 1 hour), but not that hard. After that, I really only get one spam emails ever month or two. It really does a great job at blocking a lot. It comes with the bonus of making new email addresses for those sites that you don’t trust. There is even a self-expiring address option.
The best part, is that it has a full back-up & update script system built in. Also, it runs on docker, so it won’t complain or mess up your system.
