Ec2 on Evan Hoffman

How (the hell) do you set up Splunk Cloud on Linux?

Wed, 29 Oct 2014 14:11:40 +0000

This took me way longer than I would’ve thought, mostly due to horrible documentation. Here’s my TL;DR version:

Sign up for Splunk Cloud
Download and install the forwarder binary from here.- Log in here and note the URL of your Splunk instance:

In the above picture, assume the URL is https://prd-p-jxxxxxxxx.splunk6.splunktrial.com.

Make sure your instances can connect to port tcp/9997 on your input host. Your input host is the hostname from above with “input-” prepended to it. So in our example, the input host is input-prd-p-jxxxxxxxx.splunk6.splunktrial.com. To ensure you can connect, try telnet input-prd-p-jxxxxxxxx.splunk6.splunktrial.com 9997. If it can’t connect you may need to adjust your firewall rules / Security groups to allow outbound tcp/9997

Below are the actual commands I used to get data into our Splunk Cloud trial instance:

Can I create an EC2 MySQL slave to an RDS master?

Wed, 08 Oct 2014 15:25:10 +0000

No.

Here’s what happens if you try:

mysql> grant replication slave on *.* to 'ec2-slave'@'%';
ERROR 1045 (28000): Access denied for user 'rds_root'@'%' (using password: YES)
mysql> update mysql.user set Repl_slave_priv='Y' WHERE user='rds_root' AND host='%';
ERROR 1054 (42S22): Unknown column 'ERROR (RDS): REPLICA SLAVE PRIVILEGE CANNOT BE GRANTED OR MAINTAINED' in 'field list'
mysql>

Note: this is for MySQL 5.5, which is unfortunately what I’m currently stuck with.

The m3.medium is terrible

Tue, 12 Aug 2014 13:51:08 +0000

I’ve been doing some testing of various instance types in our staging environment, originally just to see if Amazon’s t2.* line of instances is usable in a real-world scenario. In the end, I found that not only are the t2.mediums viable for what I want them to do, but they’re far better suited than the m3.medium, which I wouldn’t use for anything that you ever expect to reach any load.

Setting hostname in an EC2 instance from the name tag

Mon, 19 May 2014 15:24:09 +0000

# pip install awscli
# HOSTNAME=`aws ec2 describe-tags --region us-east-1 --filters Name=resource-id,Values=`curl http://169.254.169.254/latest/meta-data/instance-id 2> /dev/null` Name=key,Values=Name --output text --query 'Tags[*].Value'`
# hostname $HOSTNAME
# hostname > /etc/hostname

Load balancing in EC2 with Nginx and HAProxy

Mon, 09 Jan 2012 17:00:22 +0000

We wanted to setup a loadbalanced web cluster in AWS for expansion. My first inclination was to use ELB for this, but I soon learned that ELB doesn’t let you allocate a static IP, requiring you to refer to it only by DNS name. This would be OK except for the fact that our current DNS provider, Dyn, requires IP addresses when using their GSLB (geo-based load balancer) service.

Rather than let this derail the whole project, I decided to look into the software options available for loadbalancing in EC2. I’ve been a fan of hardware load balancers for a while, sort of looking down at software-based solutions without any real rationale, but in this case I really had no choice so I figured I’d give it a try.

Installing Sun (Oracle) JDK 1.5 on an EC2 instance

Wed, 16 Nov 2011 11:52:55 +0000

I’m currently working on moving a Tomcat-based application into EC2. The code was written for Java 5.0. While Java 6 would probably work, I’d like to keep everything as “same” as possible, since EC2 presents its own challenges. I spun up a couple of t1.micro instances and copied everything over, including the Java 5 JDK, jdk-1_5_0_22-linux-amd64.rpm. Installing from RPM was easy, but the EC2 instance defaults to using OpenJDK 1.6:

[root@ec2 ~]# java -version
java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.10) (amazon-52.1.9.10.40.amzn1-x86_64)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)

There were a couple of things I had to do to get the system to accept the Sun JDK as its “real” java.

Integrating Amazon Simple Email Service with postfix for SMTP smarthost relaying.

Thu, 28 Apr 2011 12:23:47 +0000

How I configured an EC2 instance to relay mail through Amazon’s SES, and some of the bumps I encountered along the way.

Relaying through Google Apps using Sendmail to bypass EC2 spam blockage

Wed, 01 Dec 2010 14:48:06 +0000

Update 3 May 2011: I’ve subsequently modified our EC2 systems to relay SMTP mail through Amazon’s SES which doesn’t have the 500 messages per day limit that Google Apps does.

A few months ago I moved a site into EC2. I didn’t want to move the existing IMAP server (ugh) so I moved the email to Google Apps. There are only about 10 mailboxes so we went with “Standard” edition (free). Once we completed the move to EC2 we discovered that emails from our webserver were bouncing due to our EC2 IP address being listed in a spam RBL. This sucked, so I looked into relaying the mail from the EC2 webserver through our Google Apps account. Fortunately this turned out to be pretty easy.

Amazon EC2 - ext3 mkfs takes 30+ minutes?

Mon, 02 Aug 2010 11:33:42 +0000

I’ve been playing around with Amazon EC2 for a new project I’m working on and so far I’m really impressed. One thing I’ve noticed, however, is that it takes forever to create an ext3 filesystem on a new volume. For example, the below command took over 30 minutes to create the filesystem on a 300 GB volume:

mke2fs -j -m0 /dev/sdf1

mke2fs 1.40.4 (31-Dec-2007) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 39321600 inodes, 78642183 blocks 0 blocks (0.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 2400 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616