Sysadmin on Evan Hoffman

Home NAS build - 2026 Edition

Thu, 05 Feb 2026 06:18:40 +0000

My journey building a new home NAS in 2026 I outgrew the one I had previously built in 2017. I discuss CPU and disk selection, benchmarking raid levels, optimizing configurations with docker and how to leverage AI assistants to simplify configurations.

Can I create an EC2 MySQL slave to an RDS master?

Wed, 08 Oct 2014 15:25:10 +0000

No.

Here’s what happens if you try:

mysql> grant replication slave on *.* to 'ec2-slave'@'%';
ERROR 1045 (28000): Access denied for user 'rds_root'@'%' (using password: YES)
mysql> update mysql.user set Repl_slave_priv='Y' WHERE user='rds_root' AND host='%';
ERROR 1054 (42S22): Unknown column 'ERROR (RDS): REPLICA SLAVE PRIVILEGE CANNOT BE GRANTED OR MAINTAINED' in 'field list'
mysql>

Note: this is for MySQL 5.5, which is unfortunately what I’m currently stuck with.

Create CloudWatch alerts for all Elastic Load Balancers

Fri, 18 Jul 2014 10:29:57 +0000

I manage a bunch of ELBs but we were missing an alert on a pretty basic metric: how many errors the load balancer was returning. Rather than wade through the UI to add these alerts I figured it would be easier to do it via the CLI.

Assuming aws-cli is installed and the ARN for your SNS topic (in my case, just an email alert) is $arn:

for i in `aws elb describe-load-balancers | grep LoadBalancerName | 
perl -ne 'chomp; my @a=split(/s+/); $a[2] =~ s/[",]//g ; print "$a[2] ";' ` ; 
do aws cloudwatch put-metric-alarm --alarm-name "$i ELB 5XX Errors" --alarm-description 
"High $i ELB 5XX error count" --metric-name HTTPCode_ELB_5XX --namespace AWS/ELB 
--statistic Sum --period 300 --evaluation-periods 1 --threshold 50 
--comparison-operator GreaterThanThreshold --dimensions Name=LoadBalancerName,Value=$i 
--alarm-actions $arn --ok-actions $arn ; done

That huge one-liner creates a CloudWatch notification that sends an alarm when the number of 5XX errors returned by the ELB is greater than 50 over 5 minutes, and sends an “ok” message via the same SNS topic. The for loop creates/modifies the alarm for every ELB.

The not-so-secret secret to Postgres performance

Mon, 07 Oct 2013 10:19:05 +0000

I manage a bunch of Postgres DBs and one of the things I almost always forget to do when setting up a new one is set the readahead up from the default of 256. I created this script and run it out of /etc/rc.local and sometimes cron it too. The 3 commands at the top are only really relevant on systems with “huge” memory - probably over 64 GB. We ran into some memory problems with CentOS 6 on a box with 128 GB ram which we ended up working around by reinstalling CentOS 5, but the /sys/kernel/mm/redhat_transparent_hugepage/ options below should fix them in 6.x (though we haven’t actually tried it on that DB, we haven’t seen any problems in other large DBs).

Goodbye, pg_dump

Thu, 26 Sep 2013 09:12:05 +0000

I’ve been a Postgres user and administrator for a while. Over the years, my views on backups have evolved.

Originally, like most people, I started out with good old pg_dump. With a reasonably small database (under 50 GB) dumping to a flat text file is a fine option. I’d generally do something like pg_dump -Upostgres dbname | gzip > dbname.sql.gz to compress it on the fly and save space. For years this seemed perfect: dumping the entire database in a single transaction into a single file that can be restored anywhere.

More fun parsing BIND query logs

Mon, 13 Dec 2010 18:21:59 +0000

“But wouldn’t it be cool if it also…” That phrase usually triggers a lot of wasted cycles in my brain, though it sometimes comes up with something neat. I added a super lame graph to the DNS QPS parser. Makes it really easy to see peaks & troughs in usage:

2010-12-12 00:00 to 00:59 => 3237, rate 0.899 queries/sec	***********
2010-12-12 01:00 to 01:59 => 2709, rate 0.752 queries/sec	**********
2010-12-12 02:00 to 02:59 => 2473, rate 0.687 queries/sec	*********
2010-12-12 03:00 to 03:59 => 2171, rate 0.603 queries/sec	********
2010-12-12 04:00 to 04:59 => 2048, rate 0.569 queries/sec	*******
2010-12-12 05:00 to 05:59 => 1918, rate 0.533 queries/sec	*******
2010-12-12 06:00 to 06:59 => 2120, rate 0.589 queries/sec	********
2010-12-12 07:00 to 07:59 => 2477, rate 0.688 queries/sec	*********
2010-12-12 08:00 to 08:59 => 2994, rate 0.832 queries/sec	**********
2010-12-12 09:00 to 09:59 => 3704, rate 1.029 queries/sec	*************
2010-12-12 10:00 to 10:59 => 4297, rate 1.194 queries/sec	***************
2010-12-12 11:00 to 11:59 => 4744, rate 1.318 queries/sec	****************
2010-12-12 12:00 to 12:59 => 5106, rate 1.418 queries/sec	******************
2010-12-12 13:00 to 13:59 => 5311, rate 1.475 queries/sec	******************
2010-12-12 14:00 to 14:59 => 5083, rate 1.412 queries/sec	*****************
2010-12-12 15:00 to 15:59 => 4855, rate 1.349 queries/sec	*****************
2010-12-12 16:00 to 16:59 => 5179, rate 1.439 queries/sec	******************
2010-12-12 17:00 to 17:59 => 4959, rate 1.377 queries/sec	*****************
2010-12-12 18:00 to 18:59 => 4693, rate 1.304 queries/sec	****************
2010-12-12 19:00 to 19:59 => 4792, rate 1.331 queries/sec	****************
2010-12-12 20:00 to 20:59 => 4799, rate 1.333 queries/sec	****************
2010-12-12 21:00 to 21:59 => 5068, rate 1.408 queries/sec	*****************
2010-12-12 22:00 to 22:59 => 4672, rate 1.298 queries/sec	****************
2010-12-12 23:00 to 23:59 => 4514, rate 1.254 queries/sec	****************
2010-12-13 00:00 to 00:59 => 3917, rate 1.088 queries/sec	**************
2010-12-13 01:00 to 01:59 => 3458, rate 0.961 queries/sec	************
2010-12-13 02:00 to 02:59 => 2874, rate 0.798 queries/sec	**********
2010-12-13 03:00 to 03:59 => 2715, rate 0.754 queries/sec	**********
2010-12-13 04:00 to 04:59 => 2690, rate 0.747 queries/sec	*********
2010-12-13 05:00 to 05:59 => 2719, rate 0.755 queries/sec	**********
2010-12-13 06:00 to 06:59 => 2831, rate 0.786 queries/sec	**********
2010-12-13 07:00 to 07:59 => 3416, rate 0.949 queries/sec	************
2010-12-13 08:00 to 08:59 => 4962, rate 1.378 queries/sec	*****************
2010-12-13 09:00 to 09:59 => 6943, rate 1.929 queries/sec	************************
2010-12-13 10:00 to 10:59 => 8296, rate 2.304 queries/sec	****************************
2010-12-13 11:00 to 11:59 => 8938, rate 2.483 queries/sec	******************************
2010-12-13 12:00 to 12:59 => 8926, rate 2.479 queries/sec	******************************
2010-12-13 13:00 to 13:59 => 8950, rate 2.486 queries/sec	******************************
2010-12-13 14:00 to 14:59 => 2173, rate 0.604 queries/sec	********

You can see pretty quickly that 4-5 AM (Eastern) is the period of lowest activity, as you’d probably expect for a US-based US-centric site. Modified perl script below.