Postgresql on Evan Hoffman

Setting user Postgres passwords via MD5

Wed, 12 Feb 2014 11:06:07 +0000

Say you want to create a Postgres account for a user but you don’t want them to have to reset it after logging in, and you don’t want to do the “come type in a password on my computer” routine. Well, here’s one way around this.

Have the user generate an MD5 of their password on their local computer. Postgres uses the username as the salt for the hash, so the command to generate the md5 on a Mac would be (assuming a username of ’ehoffman’ and a password of ‘abcdefg’):

The not-so-secret secret to Postgres performance

Mon, 07 Oct 2013 10:19:05 +0000

I manage a bunch of Postgres DBs and one of the things I almost always forget to do when setting up a new one is set the readahead up from the default of 256. I created this script and run it out of /etc/rc.local and sometimes cron it too. The 3 commands at the top are only really relevant on systems with “huge” memory - probably over 64 GB. We ran into some memory problems with CentOS 6 on a box with 128 GB ram which we ended up working around by reinstalling CentOS 5, but the /sys/kernel/mm/redhat_transparent_hugepage/ options below should fix them in 6.x (though we haven’t actually tried it on that DB, we haven’t seen any problems in other large DBs).

Goodbye, pg_dump

Thu, 26 Sep 2013 09:12:05 +0000

I’ve been a Postgres user and administrator for a while. Over the years, my views on backups have evolved.

Originally, like most people, I started out with good old pg_dump. With a reasonably small database (under 50 GB) dumping to a flat text file is a fine option. I’d generally do something like pg_dump -Upostgres dbname | gzip > dbname.sql.gz to compress it on the fly and save space. For years this seemed perfect: dumping the entire database in a single transaction into a single file that can be restored anywhere.

Problems with PostgreSQL 9.0 on CentOS 6.3 with a 2.6.32-series kernel

Tue, 25 Sep 2012 13:20:36 +0000

Note: this is cross-posted from the Yodle Tech Blog. We recently migrated our Postgres database to a significantly more powerful machine. While our existing server was doing okay load-wise, we were approaching 90% full on its Postgres volume. We’d also recently acquired an awesome new PureStorage SSD-based SAN and were excited to move our database onto it. For comparison, here are the the specs for the two systems:


	Old server	New server
Nickname	DB1	DB2
CPU	2x 6-core Xeon X5690 CPUs at 3.47GHz	2x 8-core E5-2690 CPUs at 2.90 GHz
Memory	96 GB 1333 MHz	128 GB 1600 MHz
Storage	- Dell PERC H800 SAS HBA - Dell MD1220 drive enclosure - 24x 15k RPM SAS drives configured in RAID 10	- QLogic QLE2562 dual-port 8 Gbit FC HBA - PureStorage SSD-based SAN
OS	CentOS 5.6 x86_64	CentOS 6.3 x86_64
Kernel	2.6.18-238.9.1.el5	2.6.32-279.2.1.el6.x86_64

As you can probably guess, moving from spinning disk onto SSD was the centerpiece of the upgrade. We’d tested the PureStorage system pretty extensively and used it in some secondary databases and seen some amazing improvements. We were excited to see how it would handle our production workload.

Graphing SSH dictionary attacks with HighCharts

Fri, 18 Nov 2011 14:06:03 +0000

After my 10-year-old basement Linux server died this week from a power outage, I took the sad step of giving up on it. It’s died before and I’ve patched it back together with a new power supply here or an addon PCI SATA card there, but I finally decided to throw in the towel since I had a newer old computer that had been idle for several years. The one that died was an Athlon K7 750 MHz with 512 MB ram. The new one is an Athlon 2 GHz (3200+) with 1 gig. For my uses, specs don’t really matter that much, but it’s nice to have more power for free.

33 hours to restore a Postgres DB

Tue, 26 Jul 2011 18:10:01 +0000

Restoring a huge DB takes a long time.

Using WAL archiving & Compellent snapshots for PostgreSQL backups

Wed, 13 Jul 2011 13:46:40 +0000

Changing my Postgres DB backup strategy from nightly pg_dumps to WAL archiving with SAN snapshots.

Migrating a PostgreSQL DB to a new machine without doing a dump & restore

Mon, 11 Jul 2011 14:39:16 +0000

rsync over NFS can be so fast.

PostgreSQL query to determine the largest tables in the database

Tue, 17 May 2011 10:25:32 +0000

A PostgreSQL query to see the largest tables on-disk in a given database and how large their indices are.

Do I still need swap space?

Thu, 11 Nov 2010 14:36:49 +0000

About three years ago I replaced our primary database. For years we’d been plagued by awful performance in the database and we were never able to diagnose the problem. The original server was a real beast at the time: 8 Opterons (single core), 32 gigs ram, and a fibre channel RAID connected via a QLogic HBA. This was back in 2005, so those specs don’t probably sound that impressive today, but this was a crazy configuration (with a crazy price tag to match). On paper it looked like this server should be basically invincible but the performance was awful, slowing down every process within the company. We contacted a few different companies (including CommandPrompt, which employs several of the core Pg devs) to see if they could assist us in diagnosing the problems but tuning only helped to a point. There was just something wrong with the box, maybe having to do with the FC HBA itself (which nobody knew much about).

Migrating RT from mysql to postgresql.

Thu, 06 Aug 2009 17:21:42 +0000

So I wanted to move our old RT 3.6 system to a new VM running a clean CentOS 5.3 x86 install and RT 3.8. I got RT installed but the 3.6 system was running on MySQL, which sucks (still), and I wanted to move it to Postgres, which rules (more and more each day). So Googling for “migrate rt from mysql to postgresql” brought me here. I tried the procedure listed there but it wasn’t quite working for me. my2pg.pl and mysql2postgresql.perl both caused segfaults when I tried to run them. I realized I was on my own, and it wasn’t really that bad.