Data Logging and New Gauge Design

Over the last couple of weeks I've been working on two main things for my digital gauge program. Data logging and a new gauge design (which is switchable from the S2000 design I made).

The data logging allows the system to record ALL inputs (engine data and once I have GPS, accelerometer and other sensors data from those to) to a log file and then play them back later. For instance I connected my laptop to a friends Honda Civic Type R the other day and recorded a short drive with plenty of revs (the gear indicator is wrong as it was configured for another car). I can play this recording back on my desktop PC as if I was actually in the car watching the dashboard.

The laptop I have is pretty slow so recording video of the program while it is running results in poor quality and a stuttering screen display. I can now instead simply use my more powerful desktop pc to playback this data and record the screen using some decent software.

I've included a short clip of this log playback which leads to the second part of this post. The additional design based on the Lexus LFA dashboard.

I saw the Lexus LFA on Top Gear a while back and was hooked on it's looks, I think it's probably one of the best instrument cluster designs around. My implementation of this design is not yet complete and requires some polishing off and the addition of a menu, iso symbols and a few other things but here is the video anyway...

In an earlier post I put some videos (first and second) up of the S2000 design you may have noticed that the RPM bars all change color in stages as the revs rise as a simple shift light type indicator. The LFA design also does this but the rim of the RPM gauge pulses in three stages to show when to shift...

The chassis is FINALLY here...

It's been a rocky trip for my clubman chassis. First it was delayed in England due to poor weather (i.e. the physically couldn't get the cargo to the docks!!).

It was smooth sailing to Australia from the UK and then the hassles with Customs and the Department of Transport started. This is actually quite funny and the problem seemed to be Customs and not the DOT. An application was made to Customs to have the chassis brought in as car parts (which it obviously is). However Customs decided in their infinite wisdom that it was in fact a car and demanded that I have approval in the form of a permit from the DOT to import a car.

Now these permits fall under two categories, one is for people moving back to Australia wanting to also bring back their car they've owned for 12+ months. And the second is for racing purposes, under a CAMs C3 license and with proof of what competitive races it will be used for. Both of which I am not eligible for although I could probably wrangle the one for racing. This however takes three weeks minimum and with the cargo in port and me going on my honeymoon next week, both no good...

After a many calls the DOT advised me of a Customs policy that stated a car can be imported as parts as long as there is no running gear (i.e. axles, wheels, engine and gearbox). Great! I dont have any of that stuff, so I called Customs to inform them of their OWN policy. Nope, they want it in writing from the DOT. Wtf?! Yep I triple asked and as stupid as it sounds they wanted their own policy in writing from another government body. Hilarious!

So I complied and to my dismay, they actually got upset that I apparently went over their heads to the DOT (who can overrule Customs) to have them approve it. So they decided to make things more difficult by going over everything else with a fine toothed comb (e.g. why only two items in the invoice had a price).

So after a few more emails and confirmations all was well in getting the goods released. There was one last issue in getting the goods moved to my property, the truck was too small and a crane truck was required. This delayed delivery again and meant a bit more time off from work :-(

I must say though that Ozzy the shipping agent was awesome and helped me through this process and really did go out of his way to get everything done. He even paid for the crane truck when that wasn't officially part of the included shipping costs!! You're a champ Ozzy, expect something in the mail soon...

Anyways, I've probably bored you a bit now. So I've also included some pictures of the delivery! And the rotary engine that's going to be used for the car... ~180kw, ~220nm and ~600kg The math is good...

Reading CAN data (Code Snippets)

I mentioned in a post a while back that I would post some source code from my the CANReader application I wrote.

There are a few areas of the application that were most difficult so I think they are probably the best bits to share as the rest of the program is relatively simple.

CAN Frames:
In a vehicle running a CAN bus there are a number of nodes that send and receive data to one another. These nodes are usually things like the engine computer, instrument cluster, abs computer, body control computer etc... They are much like computers on a ethernet network sending packets of data to one another via TCP. The CAN bus uses a much simpler can 11 or 29 bit data format which Wikipedia describes in detail.

The RX8 electronics and all other cars I've tested with only use the earlier 11 bit format. This format consists of an 11 bit header, a 1 bit RTR flag to identify if the message is a data request and a further 4 bits to describe the length of the message data (which can be up to 8 bytes long).

The OBDLink device that I use (which uses an ELM 327 chipset from ELM electronics) does a lot of work for me. It parses the headers for can frames and returns as an integer (i.e. base 10 not hex) along with the data bytes in hex seperated by spaces.

For example a frame might look like this: 201 1F 00 0F 00 FF EF A1 7F. 201 obviously being the header identifier and the following 8 bytes the data. Headers need to be turned on to get '201' at the beginning of the data line (use ATH1 for headers on, ATCAF0 to turn off CAN auto formatting and ATMA to get the CAN frames).

I've written a CANFrame VB.Net class that can take this data as a string and parse it into something a little more friendly. This class has two main methods NewELMCANFrame() which will take data from an ELM device and another methods NewRawCANFrame() which will take a raw CAN frame string (including the header) with each byte seperated by a string. This second method is not fully tested as I dont have anything to really test it with, but it should work with lower level devices that return full CAN frames (you might need to make it handle strings without spaces though depending on your application).

USB Serial Ports:
While developing this application I came across a nasty .Net framework bug. It seems that if you have a class uses the SerialPort class connected to a virtual com port (i.e. usb serial port) and the device connected to that port is disconnected the next time you try to read from the SerialPort it will obviously fail. This is easy to trap in a Try/Catch. However when you try to Dispose() the object and possibly open a new connection to the port the .Net framework will raise an exception that cannot be trapped by user code :-(

This is because the closing underlying stream is not handled property. This is a know issue but Microsoft seems unwilling to fix it. There is a couple of work arounds though, albeit an ugly ones. Firstly placing the following code in your app.config will fix the problem but may cause issues for you elsewhere if you have unhandled exceptions...

<runtime>
     <legacyUnhandledExceptionPolicy enabled="1"/>
</runtime> 

I've also got a wrapper class for the SerialPort named FTDISerialPort that stops the Garbage Collector to calling Finalize() on the BaseStream of the SerialPort (which is what causes the exception after the usb device is disconnected).

Dynamic Scripting Code:
One of the most useful things with the CANReader application I wrote is to in real time run a algorithm over the bits in a CAN frames data. With this I can take bit X to bit Y, cast this to a number of numerical types and then perform some math on top of that to get a required value. For instance, if byte 1 to byte 2 is the engine rpm multiplied by 4 I can take bit 0 to bit 15, convert that to an Unsigned Integer and then divide that by 4 to get the engine rpm. This can all be done while the application is running without the need to recompile.

To get the bits for all of the bytes in a CANFrame class you must iterate the DateBytes() array and use the line Dim binaryValue As String = Convert.ToString(currentFrame.DataBytes(I), 2) to convert a byte to a binary number.

I've created a separate class based on code in my CANReader application and called it ParseBits, this has a shared method called ParseBits() that you can call with all of the pertinent information to run an algorithm (in VB.Net code) on your binary number. Quite useful if you need to pull flags and other such things out of bytes.



I hope this code is useful to you guys it certainly has been to me, if it is then let me know about it!

 

TDD, NUnit, Rhino Mocks, CAN, ISO and more...

It's been a while since my last post, I would like to post with more frequency but I dont really have much to show from the last few weeks. I've written a lot more code for the digital gauge cluster which i've aptly named 'Digital Gauge', yes very exciting :-) There just isn't too much to show thats all.

Menus
I've developed a flexible fully functional multi-tier menu system that can be displayed in variety of ways depending on the displayed gauge theme (currently only a S2000 style display). This allows me to change between gauge styles, color schemes, data providers (e.g. a debugging provider returning random values for things or my OBDLink data provider for real data). I can also change a variety of settings for the application and gauges in general (e.g. redline, speed alert, audible warnings etc...) I've included some pictures of the menu in action for the S2000 style cluster below.

Indicator Lamps
You might also notice the hazard lights, I've now got a fully functional set of indicator lamps for things like the indicators/hazard lights, ABS, TCS, Engine Check, Battery etc... These can only really be powered by CAN data and not OBD data as most of these things are not accessible via OBD requests (Except for the engine check).

Unit Testing
I had been debating in my mind as to the importance and need of unit testing for a project developed by a single programmer. For small applications it isn't really needed but my application is growing at an alarming rate. At last count (two weeks ago) I had 100 individual class files each performing separate logic. With that much code there is bound to be bugs and things I've missed... So after a few discussions with some colleges at work I decided that it's worth the effort and would probably save me time in the long run by writing tests for all of my code.

Unit testing can be difficult when things are not developed to be easily testable, using techniques such as  TDD / Test Driven Development greatly simplify things. However the idea with TDD is to write the test BEFORE you write your code! So quite a bit of code missed the boat. I've also been using the MVP / Model View Presenter design pattern which lends itself well to being easily tested. The idea here is that the display, processing and data storage responsibilities are separated into their own classes. This for one makes it very easy to have multiple cluster styles as all that is required is a new view (which is based on an interface anyways).

So for new code I will use TDD where possible and for testing the existing code I've been using a combination of NUinit and Rhino Mocks. Both free testing frameworks that work in very different ways. NUinit allows is a framework that allows the creation of standardised classes for testing, because it is so widely used there are lots of free tools that allow you to execute tests from directly in the visual studio IDE. Tests can also be run from continuous integration servers (a server that constantly compiles a projects code based and executes the tests on that code when changes are detected). With NUinit tests you basically write a method that checks and compares values against expected values. Rather simple but when used correctly very effective.


Rhino Mocks on the other hand is a framework that allows classes to be mocked and stubbed. Instead of having to write stubbed implementations of class dependencies you can actually stub individual methods on mocks of those classes. So for instance if you want a method on a class dependency to return a certain value maybe from a database without having to hit the database you can simple stub that method and test the code your specifically targeting.

Sounds like a lot of work? Well it is and it isn't. It's very time consuming but it's much easier to write a test for a specific scenario than it is to find an issue on a production system with a very limited user input system (e.g. a digital instrument cluster with four buttons to drive an entire OS). So the three to four weeks of writing tests spent now should mean less head aches down the road!

Other Stuff (CAN Standards and Chassis Arrival)
I've been in contact with a few sources over the last month who have provided me with some valuable details on how CAN systems are meant to talk to each other and identifiers of standard CAN components. Once I've collated all of these details into a neat package I will make another post in regards to this as it really warrants it's own post.

My shipping agent has contacted me and my Westfield SEiw chassis has arrived at the docks. I'm still going through the customs process so it will probably be later this week when I receive the chassis. There will definitely be some photos up here when I do!

I've also had some interest in me posting some source code for the program I wrote to reverse engineer CAN messages (original post). I will make another post over the next few days with some of the code used for this program.