Tuesday, December 22, 2015

Arduino Emulator, part 2

In my last post I introduced the Arduino emulator that I wrote for CS-11M. The source code is here:

https://github.com/mike-matera/ArduinoGrader

The emulator was designed for grading and has limitations. The emulator code and the sketch run in one thread so if the sketch goes into an infinite loop the emulator hangs (unless the loop calls into Arduino code which calls the emulator). Sketches often use static initialization (see the Servo library examples) which makes emulator initialization fragile. Most of all it's tedious writing test code in C++.

Before next year I want to rewrite the emulator to make it more flexible and complete. Here are my goals for the next implementation:

  1. Write tests in Python. 
  2. Serial port interaction using Pexpect.
  3. Loadable sketches. Sketches should be loadable at run-time, rather than linked in. Loading a sketch with dlopen() let's me ensure the emulator is initialized before static initialization in the sketch. 
  4. Threading support. It will be necessary for attachInterrupt().
  5. Arduino IDE integration. It would be ideal to have the IDE compile the sketch for the emulator. That way I can integrate the emulator back-end in the same way that different chips and architectures are integrated. 
There's a significant leap in complexity. The emulator will be two processes, the C++ sketch and the Python script, communicating via a socket (and Google's protocol buffers). The emulator state that's now embodied in the Emulator class should be Python side and socket calls from the sketch should be synchronous so test code can be as precise as possible. 

But when will I have the time to build it? 

Monday, December 21, 2015

The Arduino Emulator

I needed a way to grade assignments in CS-11M, which is taught on Arduino. Downloading student's designs would have been extremely tedious. I would have had to have been strict about the circuit (exact pins, wiring, values, etc.) which I thought would have negative consequences. So I decided to build myself a simple emulator. You can see the work here:

https://github.com/mike-matera/ArduinoGrader

The emulator works at the Arduino API level. I added to it as needed to make it functional enough for the assignments I gave. Here's the anatomy of the emulator:

  emulator/


Emulator code is here. The Emulator object mediates communication between the sketches and the test code. The implementation of Arduino functions in the main namespace is in emulator.cpp. Defined:

  • micros() and millis()
  • delay() and delayMicroseconds()
  • digitalRead(), digitalWrite() and pinMode()
  • analogRead(), analogWrite() and analogReference()
  • tone() and noTone()
  • random() and randomSeed()
  • map(), etc.
These core functions internally operate on the Emulator object rather than pins or a processor-centric emulator.

  arduino/


The arduino directory contains Arduino code that was ported to the emulator. Porting proved to be easy. The serial port is emulated using stdio which handless TTYs and pipes properly. There's also a hack that lets test code create input. But it's threaded code that's not synchronized it's bound to fail in a complicated test. The arduino directory contains emulation of
  • HardwareSerial (as Serial) including Serial, Stream, Print and WString
  • Servo 
Additional libraries will have to be ported if they're used. 

  tests/ 


The tests directory contains tests for class projects. Each <sketch-name>_test.cpp is meant to test an assignment given in class where the student was asked to turn in <sketch-name>.ino. The main directory has shell scripts that glue everything together. 

Building the Emulator


Three parts are built together:
  1. Base emulator code. 
  2. <sketch-name>.ino
  3. The <sketch-name>_test.cpp 
The resulting executable runs the sketch and the test together. The build procedure is: 
  1. Gather <sketch-name>.ino and related sources. 
  2. Use the Arduino command line interface to build the sketch and leave intermediate files. This creates the <sketch-name>.cpp file.  
  3. Compile the <sketch-name>.cpp file and related source with the test code and emulator
Step 2 is important. Arduino creates function declarations that make otherwise illegal C++ legal. Further, a fail at this step tells me that the original sketch didn't compile (i.e. it's the student's fault) most compile failures in step three are my fault. 

The emulator was an indispensable but it really needs improvement. In my next post I'll discuss what I would like to do with it. 

Wednesday, November 25, 2015

The Airbnb Android App contains Spyware.

A couple of months ago I noticed that my phone was talking to China while using Wireshark on my home network. The conversation which was between my IP address and 119.147.146.70 contained this data:
POST /c/ HTTP/1.1
Content-Length: 113
Host: infoc2.duba.net
Connection: Keep-Alive
q......L......................H.7.6.............
.......................
................X..V....................HTTP/1.1 200 OK
Server: Kingsoft Web Server
Date: Mon, 28 Sep 2015 00:10:05 GMT
Content-Type: text/plain
Content-Length: 36
Connection: keep-alive
[common]
result=1
time=1443399005.
Alarming! I wasn't able to find any information about the IP address that seemed useful so, freaked out, I decided to factory reset my phone. I'm so busy during the semester that I thought it would be best to just be done with it.

Fast forward to yesterday, when after getting my IDS back online I saw this:


The RFC-1918 address listed is the internal IP address of my Android phone. So I did some more digging. I started a new Wireshark capture and watched all traffic from my phone overnight. I also installed the very handy Network Connections Android App. I paid for it too so I could capture for a longer time. What did I see? Several connections made to Chinese servers from the Airbnb app. I started to think that I was wrong to worry because it's perfectly reasonable for Airbnb to use international infrastructure (there's only one Internet, right?) Except here's a transcript of the conversation that caused the IDS alert:
POST /v2/report HTTP/1.1
Accept: application/json
Accept-Encoding: gzip
Content-Encoding: gzip
X-App-Key: 9f6627a3f8efaa87b929071c
Authorization: Basic MjgwNzY0MzQ1Mjo4ZTcyZasdfASH234yehereSAJfaA4MDc3ZTVmNDFiNQ==
Content-Length: 427
Host: stats.jpush.cn
Connection: Keep-Alive
.............j.0...E...d.......(...q.zec;
K.w..$iJ..B.....3..W........J.....q....D*2l..KL.#.5TV$..eZN......C.Jlrc...5Fk^..c.eH..>;.........q..,.&AQ..C...nK...D.N.....B...'(.4.7.%......m...%..M.. .C.f..L.. 0p....x
.o......@[...?&.o..oB......B..BY.?...o<P...}..o...>..+..8..*.........YK..a..[.k....[....
c..s.......{.?._m0.>...u.1:|E...b.(...1g..Zsl.Qp'.^......hL_..hFB..3......d.V.....s{..<.I).].LL...Yn.f.l-.._q)..
>..o&.......HTTP/1.1 200 OK
Server: nginx/1.4.0
Date: Wed, 25 Nov 2015 16:43:28 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Content-Encoding: gzip
14
....................
So what, you say? Here's what you get when you unzip the data the app sent to the server:
{"content":[{"type":"loc_info","itime":1448462400,"network_type":"WIFI,","local_dns":"10.2.5.2","wifi":[{"mac_address":"cc:b2:55:91:15:60","signal_strength":-72,"age":0},{"mac_address":"cc:b2:55:91:15:60","signal_strength":-72,"age":0},{"mac_address":"72:75:48:3e:e1:d4","signal_strength":-69,"age":0},{"mac_address":"64:0f:28:c5:2f:0a","signal_strength":-70,"age":0},{"mac_address":"cc:35:40:c8:4d:17","signal_strength":-72,"age":0},{"mac_address":"f0:99:bf:05:50:64","signal_strength":-74,"age":0},{"mac_address":"14:5b:d1:ac:f2:b0","signal_strength":-81,"age":0},{"mac_address":"08:86:3b:79:da:42","signal_strength":-84,"age":0},{"mac_address":"e4:f4:c6:01:ad:a9","signal_strength":-94,"age":0},{"mac_address":"ce:35:40:c8:4d:19","signal_strength":-73,"age":0},{"mac_address":"0a:86:3b:79:da:43","signal_strength":-85,"age":0}],"cell":[{"cell_id":00000000,"location_area_code":14950,"mobile_country_code":0,"mobile_network_code":0,"signal_strength":-111,"age":0}],"gps":[{"lat":00.000000,"lng":-000.000000,"alt":-24,"bear":0,"acc":4}]}],"platform":"a","uid":2807643452,"app_key":"9f6627a3f8efaa87b929071c","sdk_ver":"1.8.2"}
Look closely and you'll notice that the MAC addresses and the signal strength of all the clients on my WiFi as well as my local DNS server were all uploaded to the server along with my location and cellular ID (which I've blanked out with zeros). Here is the result of the Network Connections capture:

10.2.5.2 49994 58.67.203.149 7007 com.airbnb.android:10113 11/24/2015 9:27 PM
192.0.0.4 51390 119.90.34.198 7004 com.airbnb.android:10113 11/24/2015 9:29 PM
10.2.5.2 44554 58.67.196.172 7000 com.airbnb.android:10113 11/24/2015 9:41 PM
10.2.5.2 34173 58.67.196.172 7006 com.airbnb.android:10113 11/25/2015 9:01 AM
10.2.5.2 39308 58.67.196.188 7004 com.airbnb.android:10113 11/25/2015 8:51 AM
10.2.5.2 33237 183.232.42.227 80 com.airbnb.android:10113 11/25/2015 8:51 AM
10.2.5.2 38221 58.67.203.152 7004 com.airbnb.android:10113 11/25/2015 9:01 AM
10.2.5.2 43747 183.232.29.247 80 com.airbnb.android:10113 11/25/2015 9:01 AM

Airbnb does this even after you kill it's background processes. My conclusion: Airbnb is tracking my every move. I'm uninstalling the app and complaining to Google. 


Friday, May 15, 2015

Snorby on Ubuntu 14.04 LTS

I'm an IT department of one at home. It's difficult to get useful IDS tools working on your network, which is better than it used to be. Attacks are more sophisticated and easier to execute than ever. I've been experimenting with Suricata IDS and I want to see threats in a maximally useful way. Snorby is a Ruby on Rails based web application that can analyze your IDS logs and give you visibility into your network. Protection is the sum of prevention, detection and response. Log files are not detection. Snorby has a setup guide on it's website but I thought I'd make one specific to Ubuntu 14.04. There's a blog with instructions for 12.04 that will break in 14.04, as my students found out.

The key difference between the "vanilla" Snorby installation and this procedure is that I want to use Ubuntu's packaged versions of as many things as possible. I love the way Ruby bundles dependencies and compiles a standalone environment. Any sane admin would sacrifice disk space to reduce system interdependencies. I'm just seeing what I can get away with.

You must have gcc and a supporting build environment installed. I don't show that here. You install the Ruby components with:

$ sudo apt-get install ruby1.9.3 rails bundler rake wkhtmltopdf

Now you need dev packages for the gems that have C/C++ source that needs to be built:

$ sudo apt-get install mysql-server git-core libxml++2.6-dev libxslt1-dev libmysqlclient-dev

Now build. This process can be done entirely as a non-root user, therefore it should be. No excuses.

$ sudo mkdir /opt/snorby
$ sudo chown me:me /opt/snorby
$ cd /opt/snorby
$ wget https://github.com/Snorby/snorby/archive/v2.6.2.tar.gz
$ tar -xvf v2.6.2.tar.gz
$ cd snorby-2.6.2
$ bundle install
$ bundle exec rake snorby:setup

During setup I see this warning:
  "Jammit Warning: Asset compression disabled -- Java unavailable."
I'm ignoring it based on reading this thread.

Add a snorby user to the database. Don't fail to change the name and password.

mysql> grant all on snorby.* to 'snorby'@'localhost' identified by 'snorby';
Use the default configurations as a template:

cp config/snorby_config.yml.example config/snorby_config.yml
cp config/database.yml.example config/database.yml

Customize those files to match your needs. Snorby's instructions have the best information. Here's what I changed in my configuration:
  1. Basic configuration: domain, email.
  2. The location of wkhtmltopdf is /usr/bin/wkhtmltopdf
  3. Database username and password.
What's next? Snorby's site has instructions on how to start it. After that you have to integrate the output of Suricata. 

Tuesday, March 24, 2015

Embedded Linux Conference, Day 2: Winners

Here's a postcard from the Embedded Linux Conference. The clear winners of this year are the Internet of Things (IoT) and drones. There was one talk about UAVs last year and this year they have an entire track. That's something of a 20x increase in talks on the subject. Some goes for IoT which in addition to its own track also scored two of the keynote talks.

One thing that really struck me is how much needs to be done in IoT. Panasonic announced that they are open-sourcing their existing software stack. Intel is headlining their efforts in the Open Interconnect Consortium which includes Cisco as a diamond member (the highest category). Everyone is happy to announce how many people are on board but during the panel discussion the emphasis was on interoperability. Something tells me Apple Homekit will start with a huge advantage: Interoperability guaranteed. Intel presented an interesting talk about IoT security but today that is more concept than implementation.

Thursday, March 19, 2015

RA and Prefix Delegation on Ubuntu 14.04

I want my CIS-192 class to have access to public IP addresses for their projects. Unfortunately, there's just not enough IPv4 addresses for everyone. Rick setup prefix delegation on our router at school and I figured out how to integrate DHCP-PD and RA on Ubuntu. I based my efforts on the how-to here, but my network setup is quite different.

My Network

My network is loosely represented on the right. It consists of a Cisco router that is configured for SLAAC for addresses and prefix delegation for routers. Ubuntu has two interfaces. Their configuration is:

eth0: (External Interface) SLAAC
eth1: (Internal Interface) Generated by DHCP-PD from eth0.

Procedure 

Before you begin Ubuntu has to be configured to forward IPv6 packets. The following sysctl settings are required to make that work:

net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.eth0.accept_ra = 2

In order for a router to accept RAs you must set the accept_ra variable to 2. The accept_ra variable is documented here. The documentation states: 

accept_ra - INTEGER 
  Accept Router Advertisements; autoconfigure using them. 

  It also determines whether or not to transmit Router Solicitations. If and only if the functional setting is to accept Router Advertisements, Router Solicitations will be transmitted. 

  Possible values are: 
    0 Do not accept Router Advertisements. 
    1 Accept Router Advertisements if forwarding is disabled. 
    2 Overrule forwarding behaviour. Accept Router Advertisements even if forwarding is enabled. Functional default: enabled if local forwarding is disabled. disabled if local forwarding is enabled.

Since I must accept RAs to be able to use eth0 this variable must be set to 2. Here's the IPv6 configuration of /etc/network/interfaces:

iface eth0 inet6 static
address 2001:db8::1
netmask 64

iface eth1 inet6 manual 

I've replaced my actual address with the RFC-3949 documentation address. Now I need to acquire a prefix from my DHCP server. I was not able to find any documentation that explained how to make the ISC DHCP client do this. Instead I installed the WIDE DHCP client:

apt-get install wide-dhcpv6-client 

Apt has a configuration script that asks what interface the DHCP client should use for performing DHCP. Be sure to set the interface that's facing the DCHP server (that's eth0 for me). The WIDE DHCP client needs to be told to ask for a prefix. In my case that's all it asks for. 

## this is: /etc/wide-dhcpv6/dhcp6c.conf

# Send a PD request on eth0 (external facing interface)
interface eth0 { 
  send ia-pd 1;
  script "/etc/wide-dhcpv6/dhcp6c-script";
};

# Upon receiving a prefix use the 
# information to set eth1 (internal facing interface)
id-assoc pd 1 {
  prefix-interface eth1 {
    ifid 1;
    sla-id 1;  
    sla-len 0; 
  };
};

It's worth describing the contents of the id-assoc stanza above. The information comes from the manual page. The DHCP client takes the prefix and computes new prefixes and interface addresses based on the lease and the information you've given. For each interface it generates the following:

Network: 
<delegated-prefix:prefix-length>|<sla-id:sla-len>::/64-<sla-len>

Address: 
<delegated-prefix:prefix-length>|<sla-id:sla-len>|<ifid>/64-<sla-len>

The composition is a bit tricky, so let's take an example. Suppose you receive a lease for 2001:db8::/48 and you want to have up to 256 delegated subnets (i.e. sla-len is 8 bits) and had the following configuration:

id-assoc pd 1 {
  prefix-interface eth1 {
    ifid 1;
    sla-id 1;  
    sla-len 8; 
  };
  prefix-interface eth2 {
    ifid 1;
    sla-id 2;  
    sla-len 8; 
  };
  prefix-interface eth3 {
    ifid 1;
    sla-id 3;  
    sla-len 8; 
  };
};

If all worked correctly you would have the following configuration: 

eth1: 2001:db8:0:1::1/56  
eth2: 2001:db8:0:2::1/56
eth3: 2001:db8:0:3::1/56

Astute observers will note that it's not possible to assign networks with a mask greater than 64. This means that if your DCHP server delegates /64 addresses (like mine does) you do not have any bits left for your sla-id so you must set sla-len to 0. If you don't set sla-len correctly the DHCP client silently fails. Grrr.

If all the above works you should be able to see an assigned address on your internal interface. The last step is to restart radvd and have it use the new address as the basis for sending out RAs. First configure radvd to pick up the prefix that's assigned to the interface: 

## this is /etc/radvd.conf
interface eth1
{
   AdvSendAdvert on;
   prefix ::/64 
   {
        AdvOnLink on;
        AdvAutonomous on;
   };
};

The empty prefix (::/64) directive will initially cause an error because radvd starts before the prefix delegation happens. There doesn't seem to be a upstart-y way to fix that so I added the following line to the bottom of my /etc/wide-dhcpv6/dhcp6c-script:

( sleep 10 ; service radvd reload )& 

Anytime the lease is renewed radvd will reload it's configuration files. This is probably close to what you want. The radvd.conf option DecrementLifetimes should probably be set to on. That is what you're supposed to do when you receive a prefix from DHCP. I haven't tried that option yet.

EDIT: This was necessary to make wide-dhcp start at the right time:

mv /etc/rc2.d/S20wide-dhcpv6-client /etc/rc2.d/S99wide-dhcpv6-client


Sunday, March 15, 2015

Bacon Pecan Pie

I made a pecan pie for Pi day. I love pecans but I've never liked pecan pie because the overly sweet goo is such a put-off. Corn syrup is much cheaper than pecans so I guess you get what you pay for. My pies turned out right on the edge of sweet and savory. I loved them. Here's my recipe.

  • 2 lbs. whole raw pecans
  • 1 cup brown sugar
  • 1 cup maple syrup
  • 4 strips thick-cut bacon
  • 3 medium eggs
Fry the bacon until it's crisp. Cut three strips into bits and save one for garnish. In a saucepan combine the sugar and syrup with two table spoons of bacon grease and heat until the sugar is dissolved. Stir smashed pecans and bacon bits into the syrup mixture and allow it to cool somewhat then stir in the beaten eggs. I baked mine in two prefab pie crusts at 400 degrees for 40 minutes.