Category Archives: Software development

Building IA32 assembly project in Visual Studio 2017

If you saw my presentation at DevConf you may have gotten curious and want to play with assembly language yourself. Setting up Visual Studio for compiling and debugging Intel assembly is not hard, but not straightforward. Hence, this short tutorial. I’ll show you simple setup with C file as an entry point and small assembly “library”.

Project setup

1. Create new project – C++ Win32 Console Application

 

2. On a first screen click Next, as we want into pick some custom settings. On the second mark “Console Application” and “Empty Project”. For this simple example, we won’t need any default Windows headers. Click “Finish”.

 

3. Right-click on the project, “Add” -> “New Item…”. Pick C++ file, but rename it to .c, for example “source.c”.

 

4. Add another file in a similar way and also rename it – to “lib.asm”. What names you pick doesn’t really matter, as long as C and ASM file have different. Add also header file. Name it for example “header.h”. Your project should have a similar structure to this:

 

5. Right-click on the project, “Build Dependencies” -> “Build Customizations…”. Mark “masm” as selected. This adds Microsoft Macro Assembler into the build.

 

6. Right-click the assembly file (lib.asm), “Properties”. Item type most likely says “Does not participate in build”. Change it to Microsoft Macro Assembler.

 

Code

Now let’s add simplest possible code to make it run. First assembly file. This code declares simple multiplication function that takes two arguments.

;;lib.asm

.386
PUBLIC _multiply

_CODE SEGMENT dword public 'CODE' use32
ASSUME CS:_CODE

_multiply PROC near

	push    ebp
	mov     ebp, esp   
	push	ebx
	
	mov	eax, [ebp+12]			;; second argument
	mov	ebx, [ebp+8]			;; first argument
	
	imul	eax, ebx			;; multiplying; lower 32 bits -> eax; 
						;; higher -> edx
	pop	ebx
	pop	ebp
	ret
_multiply ENDP

_CODE ENDS
END

 

Then let’s add the header, so we can use this function:

#pragma once

int multiply(int a, int b);

 

And for the finish, the C file that will call our function:

#include "header.h"

int main(int argc, char *argv[]) {
	int a = 3;
	int b = 2;

	int mult = multiply(a, b);

	return 0;
}

Debugging

Now you’ll be able to debug it like every other application in visual studio. You can set breakpoints and even Watches to see what’s in registers:

 

One problem, that I’ve noticed happening quite often, is that Visual Studio doesn’t notice changes made in the .asm file and do not recompile the project. I found that a good way to force it is to change target CPU for something different, build, change back to our preferred settings and rebuild again.

Resources

Intel® 64 and IA-32 Architectures Software Developer’s Manual

This x86 Assembly guide at University of Virginia CS

 

Good luck with your experiments and exploring your computer’s architecture!

DevDay DevConf is my favourite day! Potentially…

The first programming conference I fell in love was DevDay. It really opened my eyes when I went there for the first time in 2012 and it never failed to satisfy – I wrote about it multiple times. DevDay won’t be organized this year, or maybe even won’t be organized, period. But don’t worry – Michał and Rafał (together with other awesome people) are starting a new conference. I invite you to DevConf!

DevConf logo

I sent two talks for the CFP and both have been accepted. So I’m super happy, and also a bit stressed – this is going to be the first time I’m doing two talks at the same conference. Both new. Challenge accepted!

The first talk will be related to my growing in recent years interested in Machine Learning. I’ll try to explain basics of the technicalities of training and evaluating ML models in approachable ways. You’re probably gonna be disappointed how easy it is to get a relatively good working model. I hope to get you interested enough, that you won’t surrender when the first obstacles show up.

The second talk will be about my other fascination. How computers actually work? I’ll start with what most programmers know the best these days – one of the high-level programming languages. From there I’ll explore what lays beneath, what layers built up over last few decades. We’re standing on arms of the giants of the past and it’s a good thing to appreciate it.

DevConf is held on 13-15th September in Kraków, Poland. It’s less than a 2hrs flight from most places in Europe. One day of workshop and two days of three tracks talks for very affordable price. Make sure you stay for the weekend – traditionally we have a lot of fun there also after the conference. Register here! Hope to see you there!

Phoenix, Ecto and time zones

This is another part of my series on learning Elixir.

In the previous episode, I calculated the distance of flight. I will use it later for gathering statistics. Another metric, I would like to have is a time of flight. Usually, flight departure and arrival times are presented in local time. So it gets a bit tricky to get the time length when the journey spans across few time zones.

Elixir’s standard DateTime library doesn’t work with time zones very well. The Internet suggests I should use Timex.

But first I needed to make some changes into my database because up until now, I had only flight date. A few weeks ago I wrote a post, how to update your schema with migrations and this time I followed the same steps. Still, I haven’t figure out how to do it in a bit more automated manner.

defmodule Flightlog.Repo.Migrations.CreateFlight do
  use Ecto.Migration

  def change do
    alter table(:flights) do
      modify :arrival_date, Timex.Ecto.DateTimeWithTimezone
      modify :departure_date, Timex.Ecto.DateTimeWithTimezone
    end
    
  end
end

As you can see, I used specific Timex types. This works only with Postgre, and if you want to use timezones it needs one additional step. You’ll have to add custom type to your database:

CREATE TYPE datetimetz AS (
    dt timestamptz,
    tz varchar
);

You can read more about using Timex with Ecto on this documentation page.

I also updated my flight.ex model. It looks like that right now:

defmodule Flightlog.Flight do
  use Flightlog.Web, :model

  schema "flights" do
    field :departure_date, Timex.Ecto.DateTimeWithTimezone
    field :arrival_date, Timex.Ecto.DateTimeWithTimezone
    field :flight_number, :string
    field :plane_type, :string
    field :from, :string
    field :to, :string

    timestamps()
  end

  @doc """
  Builds a changeset based on the `struct` and `params`.
  """
  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:departure_date, :arrival_date, :flight_number, :plane_type, :from, :to])
    |> validate_required([:departure_date, :arrival_date, :flight_number, :plane_type, :from, :to])
  end
end

After that, I walked along the path that’s proven to work in the flight distance part. I added a new function in my math.ex library, making use of Timex diff function:

    def flightTime(earlier, later) do
        hours = Timex.diff(later, earlier, :hours)
        minutes = rem(Timex.diff(later, earlier, :minutes), 60)
        "#{hours}:#{minutes}"
    end

And I’m calling it from the view in another function, so it’s easily accessible from the template:

  def time(time1, time2) do
    Flightlog.Math.flightTime(time1, time2)
  end

And that was it. Although it took me few hours because I was struggling a bit with Timex types. I didn’t read carefully the documentation and for example missed the step with creating new Postgre type. Good lesson here, to look into docs carefully :)

The effect:

 

Screen Shot 2017-05-07 at 23.26.43.png

As you can see, I made some approach at formatting dates. Unfortunately, I didn’t manage to show them in local time. They’re in UTC in here. This will be next step most likely.

That’s all for today. Next week we’ll try to test this new module. In the meantime, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Broadcasting custom packages on Estimote Location Beacons with Raspberry Pi

Last summer together with Karl-Henrik Nilsson and Nejc Palir I worked on a fun project that incorporated new features of Estimote Location Beacons. It was the solution for a public transport company that used beacons for two things – delivering proximity-based features and broadcasting custom messages to customers. The second part was possible thanks to GPIO ports on (then very) new Location Beacons. We worked closely with Estimote to deliver a solution based on not very well documented feature. It was built with Raspberry Pi running WiFi hotspot powered by an external battery, which runs Node.js serving configuration application. On top of that, we built client application for iPhone and several other components.

For this post, I’ll show a simpler example, to isolate one particular feature. I’ll focus on connecting Raspberry Pi to beacon, and writing code that will change broadcasted “Local Name” of it. I’ll also use Node.js as it allows for rapid changes without the need to recompile. This will be a good occasion to talk a little bit about UART, Bluetooth data frames and simple npm package we built as a result of this project. It’s open sourced on GitHub, so you can peek inside to see our lousy JavaScript and maybe offer some improvements? :)

You’ll need one Raspberry Pi with a clean install of Raspbian, one Location Beacon and 4-wire ribbon cable. Make sure your RPi can connect the Internet and install Node.js and npm.

Connecting Raspberry Pi to Estimote Location beacon

There is a bunch of features that differentiate Location Beacons from older Proximity Beacons. They have better battery life, a longer range, can broadcast multiple packets at the same time, more sensors, internal memory and few other things. They all greatly increase usage scenarios for beacons. More importantly, Location Beacons have GPIO port. You can use it for sending signals to connected devices, for example turning on lamps when you’re in proximity. But you can also use the port in UART configuration and use it to send data in both directions. For example changing BLE frame that is broadcasted by the beacon.

GPIO port has four pins. GND, Vcc IN (5V Power), GPIO0 and GPIO1Estimote pinout.png
Picture comes from official Estimote App

rpi_pinout.png
Picture comes from Element14

When you set the beacon to work in UART configuration (more on that in next paragraph), GPIO0 will be UART_RX (receiving) and GPIO1 will be ‘UART_TX’ (transmitting). To establish proper communication with Raspberry Pi, you’ll need to connect Vcc IN to Pi’s 5V power (pin 02), GND to GND (pin 06), GPIO0 (receiving) to GPIO14 (aka TXD0 – UART transmitting, pin 08) and GPIO1 to GPIO15 (aka RXD0 – UART receiving, pin 10). So power to power, ground to ground and transmitting to receiving (so one device can receive what other transmits).

Connection scheme.png

Picture: Simplified schematic made using Fritzing. The beacon image comes from a package made by Devin Mancuso that’s available on CreativeCommons Attribution 3.0 license. I modified his work to use in Fritzing.

IMG_0568.JPG

Configuring beacon

Estimote Location Beacons by default are not configured to use GPIO in UART mode. There is no option to do so in the official app, but you can use the Estimote iOS SDK to do that. Alternatively, Piotr Krawiec built a modified version of Estimote’s “Configuration” iOS example/app template that does exactly that. You need to compile it by yourself and deploy on your iPhone. Then you’ll have to connect to the beacon and configure it. That should be all. Unfortunately, you won’t get any visual confirmation from the beacon that it worked.

Screenshots from UART configuration app

Estimote Android SDK doesn’t support enabling the UART mode yet so you might want to borrow an iPhone from a colleague. Alternatively, Estimote told us that if somebody asks for this to be added to the Android SDK, they will gladly do that.

Configuring Raspberry PI to use UART instead of console

We’ll also need some changes in RPi configuration. By default, the UART ports are set to work as external console connection point. So we need to do 2 small changes:

  1. In file /boot/config.txt add line
    enable_uart=1
  2. in /boot/cmdline.txt change serial0 to serial1 and make sure baud rate is set to 115200

Writing to the beacon

UART (Universal Asynchronous Receiver/Transmitter) is a hardware used for asynchronous serial communication. It has very lightweight communication protocol and hence is quite often used in microcontrollers. Estimote beacons use UART with custom implemented protocol, called EstiUART. You can find simple documentation for it in this package from Estimote. The way it works is that whole Bluetooth frame that you want to broadcast gets wrapped in another frame of Estimote bytes and you just send it over at the certain baud rate.

Estimote has 3 custom broadcasters and you can broadcast 3 different messages in parallel. On top of that, you can use standard iBeacon or Eddystone broadcasting packages. EstiUart allows setting transmitting power, transmitting frequency and Bluetooth package for each of them separately. To set new broadcasting messages, first, you need to enable one of the broadcasters. Then you send whole BLE frame wrapped between Estimote header end stop byte.

There are 3 special bytes that you’ll have to take special care of:

  • Estimote start byte. Every message is started with it: 0x73
  • Estimote stop byte. Every message is finished with it: 0x65
  • Escape byte: 0x5C
    One example use is when you need to encode letter ‘e’ that has an ASCII code of 101 (0x65 in hex) so it won’t be confused with stop byte.

Example frame

Let’s look at the example below. It sets Local Name of the first of three advertisers to value “Test”:

0x73,0x13,0x42,0x10,0x00,0x00,0xBB,0xBB,
0xBB,0xBB,0xAE,0x02,0x01,0x04,0x06,0x09,
0x54,0x5C,0x65,0x5C,0x73,0x74,0x00,0x65

Bytes Description
0x73 Estimote start byte
0x13 Register that will be written to. 0x13 is a register to set advertised data for the 1st broadcaster. That means, that 1st broadcaster will now transmit BLE package starting from next byte. Some other examples: 0x11 is used to set advertising power (Tx) of the first broadcaster; 0x22 is used to set advertising interval for the second broadcaster.
0x42 1st byte of PDU header, describing modes of communication. This is the beginning of Bluetooth frame. For Estimote this will always be 0x42. You can read more in this great blog post.
0x10 2nd byte of PDU header – length of the packet. In this case 16. This includes following zero-byte, MAC address, following meta information and the message itself, but not padding. For the message, you shouldn’t count escape bytes, as they’re processed in estimote and won’t be transmitted as a part of Bluetooth packet.
0x00 Must be zero by design.
0x00 0xBB
0xBB 0xBB
0xBB 0xAE
MAC address. This is so-called static MAC address. It can be generated randomly but needs to meet few requirements.
0x02 Following is a number of GAP (Generic Acces Profile) sections consisting of three parts.
1) The length of the following section. In this case next 2 bytes.
0x01 2) Type of section. 0x01 means “flags”. It’s the only mandatory section. Read more here.
0x04 3) Section Content. For Estimote this flag must be set to 0x04. Read more about flags here.
0x06 Next GAP section length
0x09 Type of section. 0x09 is Complete Local Name.
0x54
0x5C
0x65
0x5C
0x73
0x74
Section Content. This is the Local Name that will be broadcasted: “Test”

ASCII for T, Escape byte (because ASCII for ‘e’ is the Stop byte), ASCII for ‘e’, Escape byte (because ASCII for ‘s’ is the Start byte), ASCII for ‘s’, ASCII for ‘t’.

0x00 Padding. From my experience, just one byte is enough.
0x65 Estimote stop byte

Beacon-pie package

To not deal with building the frame every time, me and Karl-Henrik build npm package that takes the message and do the work for us. It’s called beacon-pie. Currently, it only supports setting Local Name, but potentially you can set this way lot of other Bluetooth properties, like offered services. We also add controls to turn on/off broadcasters and change their parameters (interval and Tx).

Here is the simplest example how to set up Local Name to “Test” with BeaconPie. You can either set up all properties in one function or all of them separately.

How to confirm that it worked?

You don’t need to write your own app to just make sure it works. Eventually, you’ll want that because that’s why you make changes on the beacon in the first place. But to just check, there’s a simpler way. Download one of the beacon reading apps available on your platform application store. I use Nordic Semiconductors nRF Connect, which is available both for iOS and Android.

IMG_0769.PNGScreenshot from nRF Connect. You can see both our “device” that transmits local name ‘Test” and one of the default Estimote packets.

Summary

So to sum things up. It’s not well documented, but it’s totally possible to use GPIO of Location Beacons as UART input and alter broadcasting messages of the beacons. I used Raspberry Pi, but it will work with anything that can send bytes. And you can construct bytes manually, or use our npm package. Feel free to contribute, fork it or steal it.

Useful links

If you’re gonna write iPhone apps interacting with Bluetooth Low Energy Devices, I found those two blog posts very helpful:

  1. Interfacing TI SensorTag with iOS Devices using Swift
  2. A BLE Advertising Primer

 

Thanks for coming by. This was the most labour-heavy post on this blog so far. I also write about functional programming and machine learning. Look around, if you’re interested.

Setting up quick API with F# and Azure Functions

As mentioned in my last week Elixir blog post, I produced some quick fake API based on Azure Functions. I thought it’s gonna take a couple of minutes, but it turned out to be a whole adventure in itself.

The creating of a function is a breeze.

  1. Go to Portal, click big green “+” sign and search for “Function App”Screen Shot 2017-04-17 at 16.34.00.png
  2. Pick Function App published by Microsoft
  3. Fill all the necessary fields like App name (must be globally unique) or location. For hosting plan I used “Consumption plan” which means, I pay only for the time that function is running. I also like to pin my stuff to the dashboard, so it’s easier to find.Screen Shot 2017-04-17 at 16.37.38.png
  4. It will take several minutes to deploy.
  5. Now you can create your functions for the app. F# is hidden in small print just above “Create this function” button. So click “create your own custom function”.Screen Shot 2017-04-17 at 16.40.40.png
  6. Then with Language drop-down, pick “F#” and for Scenario – “API  & Webhooks”. There should be on the F# function triggered by HTTP request. That’s the one you want for API.
  7. You’ll get premade piece of code with a simple function that is triggered by HTTP POST with name object and responses “Hello “.

Then I started writing the logic I wanted. I made an array of hard coded airport data. I made the function to accept only GET requests (you can change it in function.json file). In code, I parse query strings and get the airport IATA code. If I have this airport in my array, I response 200 with JSON containing the data. Otherwise, I return 404. If there’s no parameter in the query string, function answers with 500.

It’s relatively simple and straightforward F# code. I just struggled a lot with debugging. The small editor on Azure doesn’t give you static analysis, nor type information and no squigglies. You need to run the function and check for compilation errors or runtime errors. There was also some weird scoping behaviour, that forced me to declare the Airports array within the function. Anyways, after 2hrs I had an API that did what I wanted. You can see the code below. It’s not bulletproof, but it does the job. And I got to play with Azure Functions a bit.

https://gist.github.com/mlusiak/5053aabbc1c6e76db082dc8daa952c81

If you want to read more about other types of F# Azure Functions, Mathias Brandewinder wrote recently two posts about timer and queue triggered functions.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Calling an external API from Phoenix app

Last week I wrote about how to extend your Ecto model. I wanted to figure it out because I thought this week, I’ll add to mother other flight related data. Turns out, I did it in completely different way. Not sure if this is the correct approach, so feel free to bash mi in comments (in a constructive way!).

One of the data I have on my flights so far is a departure and destination airport IATA code. Those are 3 letters code, commonly used in civil aviation (SFO = San Francisco, CPH = Copenhagen etc.).  I wanted to use some simple API to get more data based on this code. So I decided to write my own using Azure Functions and F#. It became a separate adventure. I’ll write another post about it.

When I had my API ready, I started figuring out, how to put it in Phoenix app. My first approach was a bit like you do stuff in ASP.NET MVC – adding those extra fields to the model, and trying to update them from within the controller. Unfortunately, I couldn’t make it work. I couldn’t even access fields of the model.

The solution I ended up doing, is putting all the web calls and logic for parsing them within the view file. It feels dirty for me. It would be very dirty in ASP.NET MVC. So my intuition is, that it’s not the best way to do it. But so far the only one I made working. After those changes my Flight view file looks like that:

https://gist.github.com/mlusiak/b00a756b891e701354cba66fae55bd78

And then I use those functions within the template that renders the view:

https://gist.github.com/mlusiak/ecb3acf26baaad25d32f78af89080319

Immediately you can see another problem with the code – for every information that I want to show, I’m making a separate web request. But it works, which is a huge improvement. Done is better than perfect.

Screenshot to prove that it does, what it should:

Screen Shot 2017-04-16 at 01.52.21.png

 

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Extending your Ecto model

Last week I used Ecto models to quickly created the database and I was very surprised how all got generated for me. But the model I build was very simplistic, and now I need to extend it. I started working on first serious functionality for the project, and I’ll have to do some changes. For start, I’m just testing by adding one field.

I generated new migrations file in /priv/repo/migrations folder and a bit by trial and error I end up with file like that:

defmodule Flightlog.Repo.Migrations.CreateFlight do
  use Ecto.Migration

  def change do
    alter table(:flights) do
      add :plane_type, :string
    end

  end
end

And after running mix ecto.migrate, I actually got some results:

Michals-MBP:flightlog michal$ mix ecto.migrate
01:01:37.663 [info]  == Running Flightlog.Repo.Migrations.CreateFlight.change/0 forward
01:01:37.663 [info]  alter table flights
01:01:37.666 [info]  == Migrated in 0.0s

This worked for adding fields to the database but didn’t automagically update all the access layers. There’s probably some way to it, but this time I did it manually.

First I updated the views, for example added following into show.html.eex:

https://gist.github.com/mlusiak/0979cb46187ee75cd724190e09aa96c1

This solved the visuals but still didn’t work. The crucial were changes in the model:

  defmodule Flightlog.Flight do
  use Flightlog.Web, :model

  schema "flights" do
    field :date, Ecto.DateTime
    field :flight_number, :string
    field :plane_type, :string
    field :from, :string
    field :to, :string

    timestamps()
  end

  @doc """
  Builds a changeset based on the `struct` and `params`.
  """
  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:date, :flight_number, :plane_type, :from, :to])
    |> validate_required([:date, :flight_number, :plane_type, :from, :to])
  end
end

I added new field both in schema part and in changeset. Cast is responsible for things being updated. I didn’t have to add it to be validates as required.

So this was actually a bit tedious, but I’m probably missing something here. Hopefully I’ll figure it out by the time I’ll need to make bigger changes.

I also found this post, that’s deal with similar problem.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Connecting to database with Ecto

In the last episode of my Elixir adventures, I messed with own-made Views and Controller to display hardcoded flight information for my FlightLog. This week I’ll try to show data based on the content of the database.

Phoenix doesn’t have built-in data access capabilities. But there’s awesome Ecto project, that’s sort of beefed up ORM. It reminds me Entity Framework (or good parts of it) in .NET or Rails. It supports multiple databases, although the default is Postgres. If you generated your Phoenix project by default and didn’t exclude Ecto, you should have everything you need to start. If no, refer to this guide

To add data access to FlightLog, I started the way, I would usually do in a web project. I installed the database (I went with Postgres), created some table, inserted some sample data. It was proven later, that it wasn’t a necessary step.

To build your first model, go to your project root and type for example:

$ mix phoenix.gen.html Flight flights date:datetime flight_number:string from:string to:string
* creating priv/repo/migrations/20150409213440_create_flight.exs
* creating web/models/flight.ex
* creating test/models/flight_test.exs
* creating web/controllers/flight_controller.ex
* creating web/templates/flight/edit.html.eex
* creating web/templates/flight/form.html.eex
* creating web/templates/flight/index.html.eex
* creating web/templates/flight/new.html.eex
* creating web/templates/flight/show.html.eex
* creating web/views/flight_view.ex
* creating test/controllers/flight_controller_test.exs

Add the resource to your browser scope in web/router.ex:

    resources "/flights", FlightController

and then update your repository by running migrations:

    $ mix ecto.migrate

As you can see, this gave me a lot of stuff – a migration, a controller, a controller test, a model, a model test, a view, and a number of templates. It also instructs as to add new Controller to the router. Let’s do that.

I also removed recent additions, because they served the same purpose. I initially though I’ll just edit them, but Ecto surprised me by doing everything for me. Updated route file looks like that:

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    resources "/flights", FlightController
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end


After that I run mix ecto.migrate which executed migration file:

defmodule Flightlog.Repo.Migrations.CreateFlight do
  use Ecto.Migration

  def change do
    create table(:flights) do
      add :date, :datetime
      add :flight_number, :string
      add :from, :string
      add :to, :string

      timestamps()
    end

  end
end


This created tables for me as defined in phoenix.gen.html. I noticed that database has some extra fields for basic journaling like inserted_at and updated_at. I logged into the database and added some data just to have something to show and fired up my project again:

mix phoenix.server

When you browse to the site, you’ll notice that not only I have a list of flights, but also buttons for actions like Edit, Delete or Add a new item. All generated for me and even with decent styling. That was a very nice surprise.

Screen Shot 2017-04-06 at 13.00.42.png

So first look at Ecto is very positively surprising. It created all the boilerplate code for me, but it is also a code that’s understandable and easy to edit. There seem not to be any underlying magic. If I need to change default behaviour I feel I wouldn’t have much problem doing that. We’ll see in future if practice will support those claims.

I haven’t covered any of the internals how Ecto works, so if you’re interested I suggest you peek into this guide and official documentation.

That’s all for today. Tune in next week for another part. Also, check previous episodesAnd if you’re interested in machine learning, look into my weekly link drop.

Adding new routes in Phoenix

This is another part of my series on learning Elixir.

Last week I mixed new Phoenix project and looked how different parts of framework fit together. This week I started adding my own routes and pages. I followed the official guide but altered it a bit to better fit my application.

Adding a Route
First things to change, when you’re adding new pages, is the router.ex file, that sits in the web folder in your project. All of the work today will be contained within web folder. To add a new route, simply add a new line within the existing scope. We’ll talk more about scopes in one of the future episodes.

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
    get "/flights", FlightsController, :index
  end

What the new Route says is, when browser hits /flights path, index action of FlightsController controller will be executed. Or in other words, this routes maps GET request for /flights to the index action of FlightsConroller. This won’t work because this controller doesn’t exist yet. Let’s fix it!

Adding a Controller
The Controller is very similar to the one I created last week. Nothing new here. We have index function that renders index.html view.

defmodule Flightlog.FlightsController do
  use Flightlog.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end
end


Adding a View and a Template
As you can see below, I added some code to the Template. I’m quite sure that it should be handled in View, but for now, I hard coded it this way. Below flights_view.ex and index.html.eex:

defmodule Flightlog.FlightsView do
  use Flightlog.Web, :view
end

https://gist.github.com/mlusiak/059a840dcf17493e50c215b54f2dcdc4

You can also see reference to the path /flights/1 that has not been declared. Let’s do it now.

Adding another Route, Controller and Template
I pasted below the code for all three files, marking changes in bold.

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
    get "/flights", FlightsController, :index
    get "/flights/:id", FlightsController, :show
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end
defmodule Flightlog.FlightsController do
  use Flightlog.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end

  def show(conn, %{"id" => id}) do
    render conn, "show.html", id: id
  end
end

https://gist.github.com/mlusiak/91f2ab3d668f89d2f8056eb0582a32ca

There were no changes in flights_view.ex.

There are some important things happening here. In Router, I pattern match against the params passed into the show function so that the id variable is bound to the value we put in the :id position in the URL. For example, if our URL is http://localhost:4000/flights/1, the id variable would be bound to 1.

In Template, I again hard coded some logic. There are for sure better ways to do it, but it works for now. It’s worth mentioning, that id passed is a string! I debugged this for an hour while comparing to integer 1 and couldn’t figure out why it doesn’t work ;). Everyday you learn something new. And the effects:

http://localhost:4000/flights/1

Screen Shot 2017-03-21 at 23.51.09.png

http://localhost:4000/flights/2

Screen Shot 2017-03-21 at 23.51.29.png

As you can see it’s all rendered within parent template. I’m not gonna get bothered by this at this point.

If you want to read more about Templates, again official guide is fantastic. I also stumbled upon this very interesting blog post that explains that Templates are just functions.

Thanks for reading this another part of my adventures with Elixir and Phoenix. Come next week for more. And if you’re interested in machine learning, look into my weekly link drop. on that topic.

First look at Phoenix

This week I decided to push some things in the project. Instead of looking into the internals of Elixir and Erlang VM, let’s actually do something. My project for DSP competition is a web application, and most popular framework for web development in Elixir is called Phoenix. Phoenix use as it’s foundation Erlang’s HTTP server – Cowboy. I’m not gonna look into it right now, I don’t think it will be necessary at this point.

More important for now will be knowledge what is Plug. It is a specification for constructing modules, that can be composed together and configure how web applications behave. Modules and functions that are built to this specification are called Plugs. Plugs can be chained together to create pipelines and can handle nearly everything, i.e. authentication or parameter pre-processing. To learn more look into this guide.

Thanks to Plugs, Phoenix doesn’t have a need for a monolithic middleware. And you can customise every step of handling a request. Let’s look at the elements that handle a request up to rendering, step by step.

The Endpoint is responsible for handling request until the moment that routes take over. Which includes starting server, applying the configuration and also applying a set of plugs that are common for all requests.

The Router is responsible for a few things. First of all, it parses incoming requests and sends them to appropriate Controller and Action. You can also create Pipelines here, which are sequences of Plugs, that can be easily applied to routes. So on top of common plugs, you can add your own. For example, to differently handle requests that will be rendered in a browser, vs. API requests.

Controllers do what they usually do in MVC model. They provide actions, which may prepare data for views, render stuff (through Views) on screen or do redirects.

Views are presentation layer. They’re rendered based on Templates. Templates in Elixir are precompiled, which makes them very fast.


Code
Armed with those basics, let’s look at some code that’s generated when you start Phoenix project.

First what I did is mix new Phoenix project instead of previous one.

$ mix phoenix.new hello_phoenix

This caused some havoc in my repository but  I managed to clean it up. After this and few other steps, like setting up a database, I could start it up.

$ mix phoenix.server

After that I could navigate to http://localhost:4000/ to see my first Phoenix application:

phoenix after start.png

Let’s take a look, how default Router looks like:

defmodule Flightlog.Router do
  use Flightlog.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", Flightlog do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
  end

  # Other scopes may use custom stacks.
  # scope "/api", Flightlog do
  #   pipe_through :api
  # end
end

You can notice declared separate pipelines for browser and API. The one for browsers has accept-headers for HTML, handles session and adds anti-forgery tokens. One for API is much simpler. Then inside of scope (which is out of the scope of this post :P) you can see this pipeline applied and route for “/” declared that will trigger PageController’s action named :index.

defmodule Flightlog.PageController do
  use Flightlog.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end
end


This renders a view based on template located in file index.html.eex.

Next week I’ll start working on the code and adjust the routes for my own needs.


More resources

If you interested into looking Phoenix yourself, their documentation is very good, especially the guides. I also watched this video from 2015 NDC Oslo by the creator of Phoenix, Chris McCord. It gave me good overview how things work in Phoenix.

Thanks for reading this another part of my adventures with Elixir and Phoenix. Come next week for more. And if you’re interested in machine learning, look into my weekly link drop.