Doing math in Elixir – calculating Great Circle Distance

In last two parts, I was setting up and consuming quick API to get additional airport data. One of this information was exact airport location on the Earth. In this part, I’ll use that information to calculate the distance of the travel and show how you can use standard math library in Erlang VM.

Calculating distance of air travel based on location is not super hard, but it needs to accommodate the fact that Earth is not flat. We’ll use something called “Great-circle distance“, as that’s how planes fly. So when you take most commonly used Mercator projection of the map, the shortest path between two points won’t be a straight line. It’s gonna have more parabolic-like shape. It’s best visible if you pick longer flights. Just go to flightradar24.com and check any plane overflying the Atlantic. They seem to be all flying towards north first and then turn south as the flight progresses. In fact, they fly a straight line, but because Eart is a sphere (in simplified model), meridians are not really parallel to each other.

After this geography primer, let’s get to math. The formula for calculating great-circle distance looks like that:

\Delta\sigma=\arccos\bigl(\sin\phi_1\cdot\sin\phi_2+\cos\phi_1\cdot\cos\phi_2\cdot\cos(\Delta\lambda)\bigr).

Where \phi _{1},\lambda _{1} and \phi _{2},\lambda _{2} are latitude and longitude of two points on Earth. This formula takes in radians and outputs radians. The result is angle difference between two points. To get the distance in kilometres, we have to multiply it by the radius of the Earth, which in the metric system is around 6370kms.

d=r\,\Delta \sigma .

To the code! I changed my functions in view, so instead of returning separately latitude and longitude, they give back a pair of coordinates as a tuple.

https://gist.github.com/mlusiak/0c934302b100a8b0f8899010fdd8422b

Then I pass the tuples to newly created function distance, that calls newly created Flightlog.Math module. I put the module in /lib folder and it all worked by the first run!

https://gist.github.com/mlusiak/031d81a94c3302bb6be9068c8bc29345

:math is a reference to Erlang VMs math library. One thing that’s needs explanation is changing degrees into radians. Radian is a different mathematical representation of the angle. 1 radian means, that the length of the arc of the part of the circle, that this degree describes equals the length of the radius of that circle. So the full circle is slightly over 6 radians (2 * Pi), as a formula for the length of the circumference of the circle is 2 * Pi * radius.

That’s all for today. Next week we’ll try to test this new module. In the meantime, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Setting up quick API with F# and Azure Functions

As mentioned in my last week Elixir blog post, I produced some quick fake API based on Azure Functions. I thought it’s gonna take a couple of minutes, but it turned out to be a whole adventure in itself.

The creating of a function is a breeze.

  1. Go to Portal, click big green “+” sign and search for “Function App”Screen Shot 2017-04-17 at 16.34.00.png
  2. Pick Function App published by Microsoft
  3. Fill all the necessary fields like App name (must be globally unique) or location. For hosting plan I used “Consumption plan” which means, I pay only for the time that function is running. I also like to pin my stuff to the dashboard, so it’s easier to find.Screen Shot 2017-04-17 at 16.37.38.png
  4. It will take several minutes to deploy.
  5. Now you can create your functions for the app. F# is hidden in small print just above “Create this function” button. So click “create your own custom function”.Screen Shot 2017-04-17 at 16.40.40.png
  6. Then with Language drop-down, pick “F#” and for Scenario – “API  & Webhooks”. There should be on the F# function triggered by HTTP request. That’s the one you want for API.
  7. You’ll get premade piece of code with a simple function that is triggered by HTTP POST with name object and responses “Hello “.

Then I started writing the logic I wanted. I made an array of hard coded airport data. I made the function to accept only GET requests (you can change it in function.json file). In code, I parse query strings and get the airport IATA code. If I have this airport in my array, I response 200 with JSON containing the data. Otherwise, I return 404. If there’s no parameter in the query string, function answers with 500.

It’s relatively simple and straightforward F# code. I just struggled a lot with debugging. The small editor on Azure doesn’t give you static analysis, nor type information and no squigglies. You need to run the function and check for compilation errors or runtime errors. There was also some weird scoping behaviour, that forced me to declare the Airports array within the function. Anyways, after 2hrs I had an API that did what I wanted. You can see the code below. It’s not bulletproof, but it does the job. And I got to play with Azure Functions a bit.

https://gist.github.com/mlusiak/5053aabbc1c6e76db082dc8daa952c81

If you want to read more about other types of F# Azure Functions, Mathias Brandewinder wrote recently two posts about timer and queue triggered functions.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Calling an external API from Phoenix app

Last week I wrote about how to extend your Ecto model. I wanted to figure it out because I thought this week, I’ll add to mother other flight related data. Turns out, I did it in completely different way. Not sure if this is the correct approach, so feel free to bash mi in comments (in a constructive way!).

One of the data I have on my flights so far is a departure and destination airport IATA code. Those are 3 letters code, commonly used in civil aviation (SFO = San Francisco, CPH = Copenhagen etc.).  I wanted to use some simple API to get more data based on this code. So I decided to write my own using Azure Functions and F#. It became a separate adventure. I’ll write another post about it.

When I had my API ready, I started figuring out, how to put it in Phoenix app. My first approach was a bit like you do stuff in ASP.NET MVC – adding those extra fields to the model, and trying to update them from within the controller. Unfortunately, I couldn’t make it work. I couldn’t even access fields of the model.

The solution I ended up doing, is putting all the web calls and logic for parsing them within the view file. It feels dirty for me. It would be very dirty in ASP.NET MVC. So my intuition is, that it’s not the best way to do it. But so far the only one I made working. After those changes my Flight view file looks like that:

https://gist.github.com/mlusiak/b00a756b891e701354cba66fae55bd78

And then I use those functions within the template that renders the view:

https://gist.github.com/mlusiak/ecb3acf26baaad25d32f78af89080319

Immediately you can see another problem with the code – for every information that I want to show, I’m making a separate web request. But it works, which is a huge improvement. Done is better than perfect.

Screenshot to prove that it does, what it should:

Screen Shot 2017-04-16 at 01.52.21.png

 

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Connecting to database with Ecto

In the last episode of my Elixir adventures, I messed with own-made Views and Controller to display hardcoded flight information for my FlightLog. This week I’ll try to show data based on the content of the database.

Phoenix doesn’t have built-in data access capabilities. But there’s awesome Ecto project, that’s sort of beefed up ORM. It reminds me Entity Framework (or good parts of it) in .NET or Rails. It supports multiple databases, although the default is Postgres. If you generated your Phoenix project by default and didn’t exclude Ecto, you should have everything you need to start. If no, refer to this guide

To add data access to FlightLog, I started the way, I would usually do in a web project. I installed the database (I went with Postgres), created some table, inserted some sample data. It was proven later, that it wasn’t a necessary step.

To build your first model, go to your project root and type for example:

$ mix phoenix.gen.html Flight flights date:datetime flight_number:string from:string to:string
* creating priv/repo/migrations/20150409213440_create_flight.exs
* creating web/models/flight.ex
* creating test/models/flight_test.exs
* creating web/controllers/flight_controller.ex
* creating web/templates/flight/edit.html.eex
* creating web/templates/flight/form.html.eex
* creating web/templates/flight/index.html.eex
* creating web/templates/flight/new.html.eex
* creating web/templates/flight/show.html.eex
* creating web/views/flight_view.ex
* creating test/controllers/flight_controller_test.exs

Add the resource to your browser scope in web/router.ex:

    resources "/flights", FlightController

and then update your repository by running migrations:

    $ mix ecto.migrate

As you can see, this gave me a lot of stuff – a migration, a controller, a controller test, a model, a model test, a view, and a number of templates. It also instructs as to add new Controller to the router. Let’s do that.

I also removed recent additions, because they served the same purpose. I initially though I’ll just edit them, but Ecto surprised me by doing everything for me. Updated route file looks like that:

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    resources "/flights", FlightController
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end


After that I run mix ecto.migrate which executed migration file:

defmodule Flightlog.Repo.Migrations.CreateFlight do
  use Ecto.Migration

  def change do
    create table(:flights) do
      add :date, :datetime
      add :flight_number, :string
      add :from, :string
      add :to, :string

      timestamps()
    end

  end
end


This created tables for me as defined in phoenix.gen.html. I noticed that database has some extra fields for basic journaling like inserted_at and updated_at. I logged into the database and added some data just to have something to show and fired up my project again:

mix phoenix.server

When you browse to the site, you’ll notice that not only I have a list of flights, but also buttons for actions like Edit, Delete or Add a new item. All generated for me and even with decent styling. That was a very nice surprise.

Screen Shot 2017-04-06 at 13.00.42.png

So first look at Ecto is very positively surprising. It created all the boilerplate code for me, but it is also a code that’s understandable and easy to edit. There seem not to be any underlying magic. If I need to change default behaviour I feel I wouldn’t have much problem doing that. We’ll see in future if practice will support those claims.

I haven’t covered any of the internals how Ecto works, so if you’re interested I suggest you peek into this guide and official documentation.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Adding new routes in Phoenix

This is another part of my series on learning Elixir.

Last week I mixed new Phoenix project and looked how different parts of framework fit together. This week I started adding my own routes and pages. I followed the official guide but altered it a bit to better fit my application.

Adding a Route
First things to change, when you’re adding new pages, is the router.ex file, that sits in the web folder in your project. All of the work today will be contained within web folder. To add a new route, simply add a new line within the existing scope. We’ll talk more about scopes in one of the future episodes.

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
    get "/flights", FlightsController, :index
  end

What the new Route says is, when browser hits /flights path, index action of FlightsController controller will be executed. Or in other words, this routes maps GET request for /flights to the index action of FlightsConroller. This won’t work because this controller doesn’t exist yet. Let’s fix it!

Adding a Controller
The Controller is very similar to the one I created last week. Nothing new here. We have index function that renders index.html view.

defmodule Flightlog.FlightsController do
  use Flightlog.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end
end


Adding a View and a Template
As you can see below, I added some code to the Template. I’m quite sure that it should be handled in View, but for now, I hard coded it this way. Below flights_view.ex and index.html.eex:

defmodule Flightlog.FlightsView do
  use Flightlog.Web, :view
end

https://gist.github.com/mlusiak/059a840dcf17493e50c215b54f2dcdc4

You can also see reference to the path /flights/1 that has not been declared. Let’s do it now.

Adding another Route, Controller and Template
I pasted below the code for all three files, marking changes in bold.

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
    get "/flights", FlightsController, :index
    get "/flights/:id", FlightsController, :show
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end
defmodule Flightlog.FlightsController do
  use Flightlog.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end

  def show(conn, %{"id" => id}) do
    render conn, "show.html", id: id
  end
end

https://gist.github.com/mlusiak/91f2ab3d668f89d2f8056eb0582a32ca

There were no changes in flights_view.ex.

There are some important things happening here. In Router, I pattern match against the params passed into the show function so that the id variable is bound to the value we put in the :id position in the URL. For example, if our URL is http://localhost:4000/flights/1, the id variable would be bound to 1.

In Template, I again hard coded some logic. There are for sure better ways to do it, but it works for now. It’s worth mentioning, that id passed is a string! I debugged this for an hour while comparing to integer 1 and couldn’t figure out why it doesn’t work ;). Everyday you learn something new. And the effects:

http://localhost:4000/flights/1

Screen Shot 2017-03-21 at 23.51.09.png

http://localhost:4000/flights/2

Screen Shot 2017-03-21 at 23.51.29.png

As you can see it’s all rendered within parent template. I’m not gonna get bothered by this at this point.

If you want to read more about Templates, again official guide is fantastic. I also stumbled upon this very interesting blog post that explains that Templates are just functions.

Thanks for reading this another part of my adventures with Elixir and Phoenix. Come next week for more. And if you’re interested in machine learning, look into my weekly link drop. on that topic.

First look at Phoenix

This week I decided to push some things in the project. Instead of looking into the internals of Elixir and Erlang VM, let’s actually do something. My project for DSP competition is a web application, and most popular framework for web development in Elixir is called Phoenix. Phoenix use as it’s foundation Erlang’s HTTP server – Cowboy. I’m not gonna look into it right now, I don’t think it will be necessary at this point.

More important for now will be knowledge what is Plug. It is a specification for constructing modules, that can be composed together and configure how web applications behave. Modules and functions that are built to this specification are called Plugs. Plugs can be chained together to create pipelines and can handle nearly everything, i.e. authentication or parameter pre-processing. To learn more look into this guide.

Thanks to Plugs, Phoenix doesn’t have a need for a monolithic middleware. And you can customise every step of handling a request. Let’s look at the elements that handle a request up to rendering, step by step.

The Endpoint is responsible for handling request until the moment that routes take over. Which includes starting server, applying the configuration and also applying a set of plugs that are common for all requests.

The Router is responsible for a few things. First of all, it parses incoming requests and sends them to appropriate Controller and Action. You can also create Pipelines here, which are sequences of Plugs, that can be easily applied to routes. So on top of common plugs, you can add your own. For example, to differently handle requests that will be rendered in a browser, vs. API requests.

Controllers do what they usually do in MVC model. They provide actions, which may prepare data for views, render stuff (through Views) on screen or do redirects.

Views are presentation layer. They’re rendered based on Templates. Templates in Elixir are precompiled, which makes them very fast.


Code
Armed with those basics, let’s look at some code that’s generated when you start Phoenix project.

First what I did is mix new Phoenix project instead of previous one.

$ mix phoenix.new hello_phoenix

This caused some havoc in my repository but  I managed to clean it up. After this and few other steps, like setting up a database, I could start it up.

$ mix phoenix.server

After that I could navigate to http://localhost:4000/ to see my first Phoenix application:

phoenix after start.png

Let’s take a look, how default Router looks like:

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end

You can notice declared separate pipelines for browser and API. The one for browsers has accept-headers for HTML, handles session and adds anti-forgery tokens. One for API is much simpler. Then inside of scope (which is out of the scope of this post :P) you can see this pipeline applied and route for “/” declared that will trigger PageController’s action named :index.

defmodule Flightlog.PageController do
  use Flightlog.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end
end


This renders a view based on template located in file index.html.eex.

Next week I’ll start working on the code and adjust the routes for my own needs.


More resources

If you interested into looking Phoenix yourself, their documentation is very good, especially the guides. I also watched this video from 2015 NDC Oslo by the creator of Phoenix, Chris McCord. It gave me good overview how things work in Phoenix.

Thanks for reading this another part of my adventures with Elixir and Phoenix. Come next week for more. And if you’re interested in machine learning, look into my weekly link drop.