14 July 2022

Buzzkill’s (Crazed)Sanity

After having used “Google Apps”–Gmail for domains–for free basically since it became available, I’m now switching to the paid version. It is now called Google Workspace.

I have a couple of domains affected by it: crazedsanity.com and buzzkill.org. They both have email, and they both have email aliases / groups.

To save money, I’m consolidating the domains into one workspace. If you have an account on either domain and wish to keep it, you should contact me, or go pay for the appropriate level on my Patreon page.

I have to pay a bunch for each account, and there’s always some administrative stuff to do. So paying for it helps me justify the cost.

The transition.

The final domain will be buzzkill.org. If you have an account on both domains, you can retrieve the CS email from the BK domain.

For your troubles, you’ll get a bunch of cool stuff:

  • Google Drive space (30 Gb)
  • Custom and secure email
  • 100 participant video meetings

Want to get more? Go for the higher next level membership! You get voting power!

Expect some outages

As with most big changes, there will likely be some delays for things like email delivery. If you use the account for chat or other things, you may experience some downtime.

Not just for existing users

Don’t have a BK or CS email? Become a member on Patreon and you can get one!

There’s other stuff, too. Check out the membership levels for more!

Category: System Administration | Comments Off on Buzzkill’s (Crazed)Sanity
31 October 2021

Why Logging Is Important

Why you need logging before you realize you’ll need it. Because when the user says, “I ran into an error…” but then fails to tell you the details, or the order of events leading up to it, and even the error itself, logging would save you.

Instead of saying, “well, let me know if you run into it again, and write down the exact error and everything you did before it happened,” you can say, “alright, I can see what happened after I just looked in the logs.”

Category: Code, held in drafts too long, Rant, System Administration | Comments Off on Why Logging Is Important
25 October 2021

Development, Automation, and Saving Time

(Note to the reader: this post has actually been a draft for a number of years.)

I’m a developer. I write code. Also, I’m just one guy with limited resources, so I can’t be spending a bunch of time testing my code in a bunch of different environments, or making sure that every single little change I make doesn’t have some unknown catastrophic ripple effect.

So I automate my life. I take advantage of freely-available tools. As much as possible.

How, you ask?

Self-Hosting Isn’t Worth It (sometimes).

First off, I don’t bother trying to do everything myself. I used to host my own code repository (a really, REALLY long time ago), and this took excess time to:

  1. setup
  2. configure
  3. maintain
  4. work with (use)
  5. keep updated
  6. handle security crap like SSL (you know, changing the website to https://….)

So I use GitHub. I’ve used SourceForge.net in the past, and it was alright for my first everyone-can-see-it code repository, but GitHub just feels nicer to work with. Going from SVN to Git was a hell of a learning curve, but totally worth it (yes, SourceForge now supports Git, but not when I switched). All my code gets out into the world, others can see it, and I get a tiny bit of exposure.

That said: self-hosting is a lot cheaper, if you’re willing to sacrifice time for money (like if you’re a poor nerd 🤓).

Continuous Integration is Very Important.

Unit testing helps avoid introducing bugs that have a nasty ripple effect. I’m not that great at getting massive code coverage with it, but I’m getting better at it.

The other thing is that, really, I can only directly (read: easily) test with the version of PHP available to me. Which is usually the newest version. And my server, or the servers that would potentially use my code, don’t necessarily have that new of a version… so I need to have something test against those old versions. That’s where Travis-CI comes into play.

So I setup the GitHub repository to work with Travis-CI. Every time I push something to GitHub, Travis-CI gets notified. They spin up a fresh new virtual server for each version of PHP that I declared my code to be compatible with. And they run all my tests in that environment. Oh, and they email me the results. WIN.

Semantic Versioning is Important.

Putting readable, easy-to-understand version numbers in the code makes life easier. Well, usually, except when I fall into “version hell,” where this project requires that project which requires another project, and none of them can decide on a version of a related project that they both like. Yuck.

Anyway, I’ll probably write more on this later. Maybe. If you’re lucky. (And I have time… Which is basically never.)

Category: Code, PHP, Rant, Software Development, System Administration | Comments Off on Development, Automation, and Saving Time
21 October 2021

Self Promotion

I was talking to one of my co-workers the other day, and the topic of self-promotion came up. I was telling him all about how he had to get better at telling others how good he is at creating digital music… and then I realized that I was horrible at it.

So this post is here for promoting me.

Hi, my name is Dan, and I’m a geek.

I’ve done a lot of geeky things.

I’m Self-Taught.

All the programming and system administration capabilities I possess have been achieved through determination. No formal education has furthered that. No classes. No certifications. (I did go to college for a while, and technically I have a 2.96 GPA, but I never finished because they failed me for bogus reasons; I think I have like three credits or six credits left to get my Associate’s degree.)

I’ve been programming since the turn of the millennium, mostly on web applications. I taught myself PHP.

I learned system administration because it interested me, and because there was a need at the place I worked. I watched the owner peck away at a console until I learned the password and gave myself access. I officially became system administrator through sheer force of will. Within a few months, I was administering two dozen servers, with all but a couple of them running various flavors of Linux.

I learned PHP through reading and experimentation.

Is there more to the story? Yes. Quite honestly, this story has been in my drafts since 2014… So it has been time to publish it for a while. So, for better or worse, here it is / was. Lol.

Category: History, PHP, System Administration, Uncategorized | Comments Off on Self Promotion
13 March 2017

Impending Server Changes

I’m going to be making some changes to the server.  I recently had my server “hacked”, which was basically just some skiddie finding a way to post one of those pharmaceutical spam ads on a couple of my WordPress sites.

Hacked - I do not think it means what you think it means.
Hacked – I do not think it means what you think it means.

I’m taking this time to figure out a couple of things.  The plan is:

  • figure out how to use Let’s Encrypt for HTTPS (for TTORP, the Story Teller Forum, and my Fitness Forum).
  • find a way to deploy + update WP sites via git (including initial setup)
  • work on a better CMS for Crazed(Sanity) sites
  • update my “deploy” system to work with GitHub and generic git (not just BitBucket.org)

That’s actually quite a bit of stuff.  It’s going to take a while to get this all setup.  It’s equally possible that I’ll post about impending downtime as I am to simply just do it: pretty much all my sites are (extremely) low traffic.  So, there, I said it.

Category: Code, PHP, Rant, Software Development, System Administration | Comments Off on Impending Server Changes
19 February 2015

Beware the five o’clock stupids

My brain gets a little “squishy” after working on something for awhile.  Pounding away at the same thing gets a little monotonous.  Over.

And over.

And over.

And over.

And over.

And oevr.

And orev.

And voer.

Adn over.

And oevr.

And OVER.

AND OEVR.

AND OEVER.

ADN VEORE.

AND OEFER.

(see what I did there?)

Wait… what do you mean you can’t find /lib/std++.so?  WTF IS THAT?  Oh… shit…

At 5:00, relative to your timezone.  Stupid strikes.  BEWARE THE FIVE O’CLOCK STUPIDS.

17 February 2015

SSH and Multiple Keys

I do a lot of stuff with SSH.  And I’m a fan of using separate keys for most things: the generic public key

~/.ssh/id_dsa.pub

for my specific workstation; and then a special, non-workstation-specific key for other stuff.

So I’ve got a key for things like:

  • Digital Ocean
  • SourceForge.net
  • BitBucket
  • GitHub

Frustratingly enough, once one has enough identities, one runs into a problem: Too many authentication failures.  Every time I see it, I have to go figure out why, and a workaround.  And that’s a couple of precious minutes that I’ll never get back.

Well, I’ve finally found a solution…  I actually already had it, I just forgot. My salvation lies in this file:

~/.ssh/config

The key is actually the first part, “Host *.hostname.com”, which is like a per-domain catch-all:

GSSAPIAuthentication no
Host *.hostname.com
    PubkeyAuthentication no
Host test
    Hostname www.test.foo
    User danf
    IdentityFile /home/danf/.ssh/digitalocean_id_rsa.pub
Host *.server04.com
    IdentityFile /home/danf/.ssh/source.server04.com_dsa.pub
Host cs
    Hostname indigo.crazedsanity.com
    User danf
    IdentityFile /home/danf/.ssh/digitalocean_id_rsa.pub
    IdentitiesOnly yes
    ServerAliveInterval 20

Oh, and this configuration allows me to use shortcuts that don’t have to resolve to real hosts and don’t require entries in my /etc/hosts file. So I can literally type ssh cs and get to where I want.

EDIT: apparently, there’s not a catch-all.  At least not a global one… but you can set one for an entire domain (like “*.hostname.com”) and that works magically.  So… close enough.  At least for government work.

Category: System Administration | Comments Off on SSH and Multiple Keys
30 January 2015

The Death of Buzzkill

Did you try to go to BuzzKill.org, and now find yourself wondering how you got here?  Probably not, unless you’re a mindless bot.  But if you’re reading this (and are not said bot), you deserve an explanation.

BuzzKill.org isn’t dead.  It’s still around, but it redirects to here.  I got tired of maintaining it–or, more accurately, fixing it when I realized it was broken–so I just redirected it to this website.  If you want to know who I am, go here.

It really wasn’t doing anything anyway.  There was a gallery, which showed some crappy old images that me and a buddy created a couple decades ago, and showed blog posts from CrazedSanity.com.  So for now and the indeterminate future, you’ve landed yourself at it’s new home.

Oh, if you’ve got yourself a spiffy email @buzzkill.org, don’t worry.  That’ll work just like before. 🙂

Category: System Administration | Comments Off on The Death of Buzzkill
21 March 2014

No Space Left On Device

I got this message while testing something on my web server. I was quite baffled to find out that there was, in fact, quite a lot of space remaining.

What Is Tail?

The “tail” program is a command line utility for Linux based machines.  To read the last few lines of a text file (such as a log), the command is “tail /path/to/file.log”.

However, the true power of the tail program comes when adding the “-f” argument.  Now, it will show the last few lines of the file, along with anything appended to the file.  Running “tail -f /var/log/nginx/error.log” shows me, in real time, the errors that are occurring on my server.

Searching For A Solution…

I read a few articles on the web about this problem, after I tried freeing up some space.

Ultimately, the problem was due to CrashPlan’s greediness (and a bug in the “tail” program).

How To Fix It… For Reals

Instead of configuring the system to have a higher value for “fs.inotify.max_user_watches”, just restart the crashplan service.  Or Dropbox, or whatever it is.  Just restart a service, then try running “tail -f” again (on virtually any file): if the error went away, then the culprit was that last service you restarted.

Category: System Administration | Comments Off on No Space Left On Device
24 February 2014

Nginx and PHP as Different Users (Pooling)

So, after having installed a few WordPress sites on my server (namely this one), I ran into some permissions errors.  I couldn’t get plugins nor any updates to install.

The problem first appeared simply as a prompt to enter FTP credentials.  That was bizarre, so I hunted to find why that was.  I realized it was a permissions issue by reading this page (among others).

I scoured the Internet for answers (in other words, I tried a lot of different search terms in Google) for a way to make Nginx, the webserver software I used, to run as the proper user.  I’d setup different users for different websites, so just changing the webserver’s default user/group wasn’t the answer.

My first solution, albeit an ugly one, was to give everybody read+write+execute permissions on my WP folders.  That was an ugly kludge, but it worked.  And so it sat for some time.

Then I finally found how.  Through some bit of serendipity, I found an article on Apache and suExec.  I changed the term to Nginx with suExec, and found the answer… sort of.

Nginx, PHP-FPM, and Pooling

So the key was the “pooling” part of PHP-FPM that I’d basically ignored.  I had read the configuration file, but didn’t really understand it.

But after reading this article about pooling with Nginx and PHP-FPM, I found the answer.

So PHP-FPM can be configured to run different pools.  Basically, that means that there are multiple main processes for PHP, and they can run as different users.

Easy.  Added a new pool, changed it’s name and the user, and the new process (well, processes) appeared, with the correct user.  But how could I attach that to my website, so it ran as the correct user (instead of www-data)?

It’s All In The Socket

The bit of magic that makes Nginx hand off the PHP work to the correct pool is the socket.  The new pool needed to have a unique socket, then the affected websites needed to be reconfigured to use the socket corresponding to the appropriate pool.

I went back and changed my new pool to have a unique socket name, then restarted the php5-fpm process.  I then went and changed my website’s configuration file to use the corresponding socket.

Before restarting Nginx, I changed the permissions on my website’s folder to no longer be world readable/writable.  Then I attempted to delete an old plugin: as expected, I got a permissions error.  Restarted Nginx, then tried again, and it worked.   Woot!

Category: Nginx, PHP | Comments Off on Nginx and PHP as Different Users (Pooling)