Exq: The background job solution for Phoenix

Although Elixir and Phoenix is super fast, there are still features that will require a background job.

Exq (https://github.com/akira/exq) has been a pleasure to use and was easy to setup.

Redis

Exq like Sidekiq uses Redis as the store for the job queue. The easiest way to setup Redis on a Mac is to use homebrew.

$ brew install redis

Install Exq

You will need to add Exq to your mix.exs file.

# /mix.exs
  defp deps do
    [
      # ... other deps
      {:exq, "~> 0.2.1"}
    ]
  end

Be sure to get your new dependency with mix.

$ mix deps.get

Configure for development

# /dev.exs
...

config :exq,
  host: '127.0.0.1',
  port: 6379,
  namespace: "exq"

Start Exq

You can start exq manually with:

{:ok, _} = Exq.start_link

But most likely you will want exq to start with your Phoenix app. You can do this by start exq as another OTP application.

# /mix.exs
  def application do
    ...
     applications: [:phoenix, :phoenix_html, :cowboy, :logger,
                    :phoenix_ecto, :postgrex, :exq]]
  end

Create worker

Our first worker will just add a record to our database. In my test project, I am going to create a Widget.

I will first have Phoenix generate my migration and model.

$ mix phoenix.gen.model Widget widgets name:string
$ mix ecto.migrate

Now to create the worker.

defmodule YOUR_APP.CreateWidgetWorker do

  def perform do
    %YOUR_APP.Widget{name: "Exq Test"} |> YOUR_APP.Repo.insert!
  end

end

Queue worker

To queue our worker, we will use the iex console.

$ iex -S mix

Then we will queue our worker.

> Exq.enqueue(:exq, "default", "YOUR_APP.CreateWidgetWorker", [])

The parameters for enqueue are: (namespace, queue, worker, args)

Here are my simplified console results:

> YOUR_APP.Widget |> YOUR_APP.Repo.all
[]

> Exq.enqueue(:exq, "default", "YOUR_APP.CreateWidgetWorker", [])
{:ok, "7c38f006-04cc-453f-9cb9-22613b2ab5d0"}

# might have to wait like... a millisec. Give the job time to process.

> YOUR_APP.Widget |> YOUR_APP.Repo.all                               
[%YOUR_APP.Widget{__meta__: #Ecto.Schema.Metadata<:loaded>, id: 1,
  inserted_at: #Ecto.DateTime<2015-08-30T21:43:44Z>, name: "Exq Test",
  updated_at: #Ecto.DateTime<2015-08-30T21:43:44Z>}]

Setup on Heroku and Redis

Currently exq does not support using a url for redis configuration so you will need to set ENV values for your redis host, port, and password.

Under the cover exq is making calls to an Erlang library. It is important to convert the String "" read from your ENV variablues to a char_list ''.

We also need to convert the port to an integer.

config :exq,
  host: String.to_char_list(System.get_env("REDIS_HOST")),
  port: String.to_integer(System.get_env("REDIS_PORT")),
  password: String.to_char_list(System.get_env("REDIS_PASSWORD")),
  namespace: "exq"