Feedbin Supports JSON Feed!

by Ben Ubois

Feedbin now supports subscribing to JSON Feeds.

JSON Feed is an alternative to the RSS/Atom formats. The great thing about JSON Feed is that it encodes the content as JSON instead of XML. This is good because parsing and writing XML feeds is hard.

The specification has a small surface area and is a great piece of technical writing. You should check it out. If you publish a website, consider offering a JSON Feed alongside your RSS feed.

One of the criticisms I’ve seen of JSON Feed is that there’s no incentive for feed readers to support JSON Feed. This is not true. One of the largest-by-volume support questions I get is along the lines of “Why does this random feed not work?” And, 95% of the time, it’s because the feed is broken in some subtle way. JSON Feed will help alleviate these problems, because it’s easier to get right.

I also want JSON Feed to succeed because I remember how daunting RSS/Atom parsing were when building Feedbin. If JSON Feed was the dominant format back then, it would have been a non-issue.

Testing Huge PostgreSQL Backups for 37 Cents a Day

by Ben Ubois

I spent some time revisiting and improving Feedbin’s backup strategy recently.

I was prompted by a couple of high-profile database incidents including GitLab and Instapaper.

I already felt pretty good about the backup system I had in place. I had tested backup recovery manually and felt the tools were solid.

However, if for some reason the database backup was failing or my restore procedure had a mistake, I would not know until I actually had to recover the database.

Feedbin uses WAL-E from Heroku to continually back up the database. WAL-E, and the PostgreSQL architecture that enables it, gives you offsite backups that can be recovered to any specified point in time.

WAL-E is set to back up the full database every day via Cron.

wal-e backup-push /var/lib/postgresql/9.2/main

This command pushes the database to S3. This functions as a base backup that when combined with the WAL archives, that are continuously uploaded, can be restored to any point after the base backup started.

WAL-E offers a counterpart command, backup-fetch, to actually restore the data from a backup-push. To test the backups I needed to build in an automated way to restore the database.

Feedbin already uses Digital Ocean for a few things, so my first thought was to use Digital Ocean for this. I wrote a script to provision a Digital Ocean server, restore the backup to it and then delete the server after the backup completed.

This turned out to be too expensive. Sending data to S3 is free, but reading it back out will cost you. For Feedbin’s database this worked out to be about $40 every time I restored the database. I wanted to test backups daily but the data transfer cost would quickly add up to about $1,200/month.

While I was looking at S3 pricing, I found out that reading from S3 is free when it is read by an EC2 instance in the same region as your S3 bucket. It was also possible to save money on the EC2 instance itself by using Spot Instances instead of on-demand. With Spot Instances you bid on your instance and AWS tells you if you can have it for that price or not.

Critically, no matter what your bid is, you never pay more than the spot price which is “The current market price of a Spot instance per hour.” With this in mind you don’t have to guess what to bid and since your bid matches the on-demand price, your instance will never be terminated early due to the price exceeding your bid.

The instance I want costs $0.78/hr so I bid $0.78/hr, but only end up paying the spot price of ~$0.18/hr.

Price of database restore with On-Demand vs. Spot Instances

I was new to the AWS CLI, but once I figured out the right data to send, it turned out to be a fairly simple script.


export AWS_SHARED_CREDENTIALS_FILE=/etc/aws.d/credentials.ini

user_data=$(base64 --wrap 0 /etc/aws.d/pg_restore)
launch_specification=$(cat <<_EOF_
    "UserData": "${user_data}",
    "ImageId": "ami-6edd3078",
    "SecurityGroupIds": ["sg-XYXYXY"],
    "InstanceType": "c4.4xlarge",
    "SubnetId": "subnet-XYXYXY",
    "EbsOptimized": true,
    "BlockDeviceMappings": [
            "DeviceName": "/dev/sda1",
            "Ebs": {
                "DeleteOnTermination": true,
                "SnapshotId": "snap-0a7be01f9503d7bee",
                "VolumeSize": 800,
                "VolumeType": "gp2"
    "IamInstanceProfile": {
        "Name": "database-restore"

/usr/local/bin/aws --region us-east-1 \
ec2 request-spot-instances \
--spot-price "0.78" \
--instance-count 1 \
--type "one-time" \
--launch-specification "${launch_specification}"

This puts in a request to launch a c4.4xlarge instance with 800GB of storage. It also specifies UserData which is executed by the instance after it boots up, which is a perfect fit for the script that configures the machine to run postgres and restore the database backup.


# pg_restore

# Add postgresql.org offical releases as an apt source
sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
apt-get install -y wget ca-certificates
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

# Install dependencies
apt-get -y update
apt-get install -y postgresql-9.2 postgresql-contrib-9.2 postgresql-server-dev-9.2 \
build-essential python3-dev python3-pip libevent-dev daemontools lzop pv ssl-cert

# Make the postgres user a sudoer so it can shut down the machine later
echo "postgres ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/postgres
chmod 440 /etc/sudoers.d/postgres

# Install and configure WAL-E
python3 -m pip install wal-e[aws]
mkdir -p /etc/wal-e.d/env
echo "secret-key"   > /etc/wal-e.d/env/AWS_SECRET_ACCESS_KEY
echo "access-key"   > /etc/wal-e.d/env/AWS_ACCESS_KEY_ID
echo 's3://bucket/' > /etc/wal-e.d/env/WALE_S3_PREFIX
chown -R root:postgres /etc/wal-e.d

# Download the latest backup
service postgresql stop
rm -Rf /var/lib/postgresql/9.2/main
envdir /etc/wal-e.d/env /usr/local/bin/wal-e backup-fetch --pool-size=16 /var/lib/postgresql/9.2/main LATEST

# set the postgres recovery settings
sudo -u postgres bash -c "cat > /var/lib/postgresql/9.2/main/recovery.conf <<- _EOF_
restore_command = 'envdir /etc/wal-e.d/env /usr/local/bin/wal-e wal-fetch --prefetch=16 \"%f\" \"%p\"'
recovery_end_command = 'mail -s \"Database Restore Complete\" email@example.com && sudo shutdown -h now'

service postgresql start

This is all that is needed to stand-up a fully functioning PostgreSQL server and restore the database. No Chef, Ansible or any other provisioning tools required.

The important part here is that postgres lets you specify a command to run once recovery is complete, the recovery_end_command.

recovery_end_command = 'mail -s "Database Restore Complete" email@example.com && sudo shutdown -h now'

Here I have it send me an email and shut down the server, which terminates the EC2 instance so it’s no longer incurring cost.

If the email goes missing, then I know the restore never completed and I can go figure out what went wrong. AWS helps you out here too. The results of the UserData script are automatically logged to /var/log/cloud-init-output.log So you can seen exactly where the restore went wrong.

I would be interested in hearing any questions or comments about this.

Subscribe Improvements

by Ben Ubois

There’s a great new interface for subscribing to feeds. The process of adding a new feed has been streamlined by giving you the ability to rename and tag feeds while subscribing.

Before, subscribing, renaming and tagging were separate operations. You had to click around a lot to get your new feed in the right place with the right name.

Also, you can now see a favicon and the feed URL for what you are about to subscribe to. And the whole thing is easy to navigate using just the keyboard.

Feedbin on The Changelog Podcast

by Ben Ubois

I was a guest on The Changelog podcast, hosted by Adam Stacoviak and Jerod Santo. Thanks guys!.

I’m happy with how it came out. I think it’s worth your time to listen if you’re at all interested in Feedbin, the business of RSS or Feedbin internals.

Regarding Readability

by Ben Ubois

Readability is shutting down and that means making a few changes to Feedbin.

Readability offered two services both named Readability.

  1. A read-it-later/Instapaper type of service. Feedbin offered an integration to let you easily add links to your Readability account. This has been removed from Feedbin.
  2. A parser API. Feedbin used this service to provide the full content of partial-content feeds. This functionality will continue to exist in Feedbin, but powered by Diffbot.

For now I’ve chosen Diffbot to fill in for the functionality that Readability’s parser API provided. Diffbot’s data is great. The whole company is focused on offering a suite of products that extract useful information from webpages and has a simple subscription business model so I’m optimistic that it will only improve.

I looked at a few alternatives to Diffbot, including some open-source projects and Mercury. Ultimately Diffbot’s solid data and presence of a business model made it the easy choice.

I’m planning on leaving the Readability icon in-place. Readability’s parser functionality is tough to convey in an icon, and I think that taking advantage of the brand recognition of Readability makes sense for now. Also, I like to think of it as a small homage.

Readability was a great product that Feedbin relied on for years. It will be missed!


It turned out that DiffBot did not offer the performance necessary for this feature. It’s been updated to use Mercury Web Parser. Also the icon has been updated to:

Feed Toolbar

by Ben Ubois

Feed Toolbar

The feed toolbar makes it easy to perform common feed actions including:

  1. Renaming
  2. Tagging
  3. Unsubscribing

The feed toolbar can be accessed by clicking on the feed title above the article.

This way when you’re browsing unread articles you don’t have to go hunting for a specific feed in the sidebar when you want to make a change.

Finally, the feed toolbar creates a way to rename feeds on a mobile device.

Subscribe to Email Newsletters in Feedbin

by Ben Ubois

You can now receive email newsletters in Feedbin.

Newsletter Subscriptions

To use this feature, go to the settings page and find your secret Feedbin email address. Use this email address whenever you sign up for an email newsletter. Anything sent to it will show up as a feed in Feedbin, grouped by sender.

Reading email in an email app feels like work to me. However, there’s a certain class of email that I want to enjoy reading, and Feedbin is where I go when I want to read for pleasure.

For example, many great websites offer subscription content, usually with an email newsletter component. Not only do I enjoy the premium content from these sites, but I believe this a great way forward for people to support writers. Personally I have paid subscriptions to four of these including:

This feature is also great for mailing lists and product announcement emails and since it’s just a regular feed, it will sync with your favorite native app as well.

Feedbin Notifier vs. Notify by Facebook

by Ben Ubois

Today, Notify by Facebook was released. Conceptually, this is a similar app to Feedbin Notifier, which was released two days ago. The timing is coincidental but still interesting.

Left: Feedbin Notifier. Right: Notify by Facebook.

A similar coincidence happened when Feedbin launched, which is that two days later Google announced they were shutting down Reader. Weird timing.

In my mind, Feedbin Notifier offers many advantages over Notify. The biggest reason to choose Feedbin Notifier is that it works with any source that offers an RSS feed, while Facebook has a limited number of built in sources.

Feature Feedbin Notifier Notify by Facebook
Sources Anything with an RSS feed 72
Apple Watch App Yes No
Read offline Yes No
Spotlight Integration Yes No
Sync Yes No
Privacy Yes LOL
Price $3/mo Free

This comparison is obviously biased. However, it is worth pointing out that Notify costs nothing, while Notifier only works with a paid Feedbin subscription.

I think that having a large free competitor like this validates the idea. It also creates a market for a premium, more fully-featured version, which is what Feedbin Notifier is. Competing with Free is nothing new, it’s what Feedbin has been doing since day one.

Feedbin Notifier for iPhone, iPad and Apple Watch

by Ben Ubois

There are already many great full-featured apps that work with Feedbin. Feedbin Notifier aims to be different.

Feedbin Notifier is a notifications based reader. The idea is to select the handful of feeds or keywords you care about most. Then when Feedbin matches an article, it will send a push notification to your iPhone, iPad or Apple Watch, keeping you informed throughout the day.

This way, Notification Center becomes the primary interface for catching up on the stories that are important to you. You’ll see articles along side your email and other notifications allowing tell at a glance if there’s anything you want to read right away.

Feedbin is a free universal app for iPhone, iPad and the Apple Watch. On the Apple Watch you can read full articles right away or for a better reading experience use Handoff to continue reading on your iPhone or iPad.

Hope you enjoy it! Would love to hear your feedback.

Image Previews

by Ben Ubois

Feedbin now features image previews in the center column.

Feedbin, now with more images.

The most important thing to me when building this feature was that only images that are reasonably high quality would show up here. To do this Feedbin uses a set of criteria that an image must meet in order to be chosen.

Feedbin can find a variety of images including:

One of my favorite operations that Feedbin does to ensure the quality of these image previews is facial detection using OpenCV. By getting a rough idea about where faces in an image might be, Feedbin is able to get a better crop. I first saw this idea used in Twitterrific and loved the results.

Crop with face detection. Sample image from The Great Discontent.
Crop without face detection. Sample image from The Great Discontent.

Adding images is a big visual change and if you prefer the old look you can turn off image previews in the Appearance settings.