[PencilGeek]

Lately,

I've been working on a little reverse engineering project. I bought a CAN bus analyzer and decided to hook it up to the car's ECU PT-CAN data bus. I'm working with the BMW E92 M3 which uses the MSS60 ECU. This PT-CAN data bus is the computer bus that the ECU uses to receive data from other car computer modules and used to broadcast data from itself to other car computer modules. (PT-CAN = Power Train CAN bus -- attached to the ECU.) I wanted to see how many engine sensors and gauges I could find on the PT-CAN bus, and how useful they would be for creating a set of virtual gauges for use with a Video VBox. Here's a preliminary version that I helped some guy prepare. This video shows the values supplied by Racelogic as part of their Video VBox product. I wanted to see how far I could expand what they've found and add even more "virtual gauges" to the Video VBox. I also wanted to know why their throttle position virtual gauge had a bug in it. But in order to fix it, I needed to find it myself and figure out what was wrong with their version.

Video vBox CAN Bus Example with donuts and 60-130 - YouTube

So far, I've found that the ECU has 81 different variables that it broadcasts and recieves on the PT-CAN data bus. The variables range from two to eight bytes wide. The 8-byte variables may (and often times do) contain more variables inside it. So I'm not just looking at 81 different variables, I'm probably trying to reverse-engineer about 240-250 different variables.

In addition to these 81 different variables, the PT-CAN data bus is also used by the Bavarian Technic ("BT") Tool to send and receive it's data to the ECU.

Phase-1: Capture CAN bus traces
Using the CAN analyzer, the first order of business is to capture sizeable bus traces that can be stored and analyzed later. So I captured two 16-minute traces of the car starting and driving. I captured this data while the BT Tool was also connected and sending/receiving it's own data and data logging at the same time.

My data logs were limited to approximately 16-minutes each. The limitation was due to the amount of samples that the CAN analyzer would store at one time. The PT-CAN is sending/receiving approximately 1000 messages per second, and the CAN analyzer tool I purchased can store about 1,000,000 samples before it becomes unhappy. 1M samples is about 16-minutes of car running -- which is ample time to capture data to catch the car warming up and driving around for a little while.

Phase-2: Reverse engineering BT Tool protocol
The second order of business was to reverse-engineer the BT Tool protocol. While I was capturing the CAN bus traces, I was also data logging with the BT Tool RPM, various engine temperatures, gear selection, etc. By reverse-engineering the BT Tool protocol, I'm hoping to use the data sent to the BT Tool to find other values on the ECU PT-CAN bus.

Reverse engineering the BT Tool protocol was pretty easy. The BT Tool always sends data on one address and it receives data on another. The difficulty is figuring what it's sending and receiving. By doing a little experimentation, it didn't take long before I had the entire BT Tool protocol figured out. I know know exactly what commands the BT Tool is sending, and I know how to interpret the data it is receiving.

Phase-3: Figure out what's meaningful and what isn't
The third order of business was to figure out everything that's possible to figure out -- in a reasonable amount of time. I wrote a few computer programs to parse the 1-million sample data logs, sort the data, and figure out what's changing and what isn't. I created a histogram of the PT-CAN bus to see this information very clearly. This would be helpful to me to figure out what I wanted to look at, and what I could safely ignore.

Here's the PT-CAN bus histogram:

ID(xxxxxxxx):  B1, B2, B3, B4, B5, B6, B7, B8
ID(000000A8):  FF, FF, FF, F1, FF, FF, 0F, 63
ID(000000A9):  FF, DF, FF, FF, 7F, FF, FF, FF
ID(000000AA):  FF, FF, FF, FF, FC, 7F, B4, FF
ID(000000B6):  FF, FF, FF, FF, 2F, 00, 00, 00
ID(000000C4):  FF, FF, FC, FF, FF, FF, F1, 00
ID(000000C8):  FF, FF, F0, FF, FF, FC, 00, 00
ID(000000CE):  FF, FF, FF, FF, FF, FF, FF, FF
ID(00000130):  55, 50, E8, F5, FF, 00, 00, 00
ID(00000135):  F5, FF, 00, 00, 00, 00, 00, 00
ID(00000175):  FF, 00, 00, 00, 00, 00, 00, 00
ID(00000194):  FF, FF, 00, FC, 00, 00, 00, 00
ID(0000019E):  02, FE, FF, FC, FE, 63, 3F, FF
ID(000001A0):  FF, B3, FC, FF, FF, FF, FF, FF
ID(000001A3):  FF, 5F, FE, F2, FF, 00, 00, 00
ID(000001A6):  FF, FF, FF, FF, FF, FF, F0, FF
ID(000001B4):  FF, D3, E2, FF, 56, 77, FE, FF
ID(000001B5):  C8, FF, 1F, 01, CA, FF, FF, 00
ID(000001B6):  FF, FF, 3C, 00, FF, FE, 7F, FF
ID(000001D0):  FF, FF, 7F, CF, FF, FF, 0D, FF
ID(000001D6):  C0, 0C, 00, 00, 00, 00, 00, 00
ID(000001D9):  FF, 3F, FF, 00, 00, 00, 00, 00
ID(00000200):  41, 91, C0, F7, FF, FF, FF, FF
ID(00000202):  FE, FF, 00, 00, 00, 00, 00, 00
ID(0000021A):  80, 33, F7, 00, 00, 00, 00, 00
ID(00000226):  00, FE, FF, 00, F3, 00, 00, 00
ID(0000023A):  00, 33, 00, 60, 00, 00, 00, 00
ID(00000242):  11, F1, FF, FF, FF, 00, 00, 00
ID(00000252):  CE, FF, 00, 00, 00, 00, 00, 00
ID(000002A6):  00, F8, 00, 00, 00, 00, 00, 00
ID(000002B2):  3F, 3F, 3F, 3F, FF, FF, 1C, 0F
ID(000002B3):  FF, FF, FF, F6, E7, 00, 00, 00
ID(000002C0):  FF, EB, FF, 00, 00, 00, 00, 00
ID(000002CF):  F9, FF, 00, 00, 00, 00, 00, 00
ID(000002D2):  FF, FF, FF, 00, 00, 00, 00, 00
ID(000002F1):  FF, FF, FD, 00, 00, 00, 00, 00
ID(000002F3):  0F, FC, FF, 00, 00, 00, 00, 00
ID(000002F6):  00, F5, 00, 00, 00, 00, 00, 00
ID(000002F8):  0F, 3F, 3F, 0C, 2F, DC, 07, FD
ID(000002FA):  FD, 05, FF, FF, FF, 00, 00, 00
ID(000002FC):  A3, 00, 0C, FF, FF, FF, FF, 00
ID(00000310):  6F, FF, FF, 03, 00, 49, 11, 00
ID(00000311):  00, F0, 00, 00, 00, 00, 00, 00
ID(00000315):  FF, F1, 00, 00, 00, 00, 00, 00
ID(0000031C):  00, E0, 7F, 67, 6F, 6F, FF, FB
ID(00000322):  FF, 07, FF, 07, FF, 07, FF, 07
ID(00000326):  F8, FF, 00, 00, 00, 00, 00, 00
ID(00000330):  FF, 8E, 00, 2F, 3F, 3F, FC, 0F
ID(00000332):  FF, FF, 00, 00, 00, 00, 00, 00
ID(00000348):  FE, FE, FE, 00, FE, FE, FF, 00
ID(0000034A):  FF, FF, FF, 7F, FF, FF, FF, 7F
ID(0000034C):  FF, 7F, FF, 7F, FE, 00, F0, FF
ID(0000034E):  FE, FF, 1F, 02, 60, 00, FE, FF
ID(0000034F):  FF, FF, 00, 00, 00, 00, 00, 00
ID(00000374):  E7, E7, FF, 00, FD, 00, 00, 00
ID(00000380):  50, 33, 36, 31, 37, 33, 38, 00
ID(00000381):  7D, FF, 00, 00, 00, 00, 00, 00
ID(00000383):  C0, FF, 00, 00, 00, 00, 00, 00
ID(00000388):  0E, 02, 68, 14, C0, 02, 28, 00
ID(00000395):  08, 08, 50, 64, FF, 00, 00, 00
ID(00000399):  FF, 51, 25, 2A, 81, 97, 00, 00
ID(000003A9):  F2, FF, 00, 00, 00, 00, 00, 00
ID(000003AC):  FE, DF, 00, 00, 00, 00, 00, 00
ID(000003B0):  FF, FF, 00, 00, 00, 00, 00, 00
ID(000003B3):  11, FF, FF, 7F, 00, F8, 00, 00
ID(000003B4):  FF, F3, B9, FD, FF, FF, FF, FF
ID(000003B9):  FF, F1, F8, 00, 00, 00, 00, 00
ID(000003BE):  FE, FF, 00, 00, 00, 00, 00, 00
ID(000003EF):  FF, FF, 00, 00, 00, 00, 00, 00
ID(00000480):  12, 42, FE, 01, FF, FF, FF, FF
ID(00000492):  29, 52, FF, 01, FF, FF, FF, FF
ID(000004A9):  39, 52, FE, FF, FF, FF, FF, FF
ID(000004B9):  00, 52, FF, FF, FF, FF, FF, FF
ID(00000580):  6F, 1F, C1, FF, FF, FF, FF, FF
ID(00000592):  F2, 33, D7, FF, FF, FF, FF, FF
ID(000005A9):  73, D7, 80, FF, FF, FF, FF, FF
ID(000005C0):  82, 31, 67, F1, 3C, FA, 94, 62
ID(000005D6):  6E, 01, 00, 00, FF, FF, FF, FF
ID(000005E0):  8D, 03, FF, FF, FF, FF, FF, FF
ID(000005F2):  6E, 1B, 00, 01, FF, FF, FF, FF
ID(000005F8):  08, 08, 00, 00, FF, FF, FF, FF

Phase-4: Categorize the variables
Now it's time to figure out what's meaningful data and what isn't. This is where the job gets a lot more difficult. It's time to start looking at the data and figure out what is in there. This isn't an easy task.

First I need to figure out if the data is changing or not. Next I need to figure out if the data represents a changing analog value or is a set of binary on/off flags. I need to figure out where the data starts and where the data ends (how big it is). Then I need to figure out whether the data appears in forward order or some type of reverse order.

During this process I realized BMW engineers are some real sneaky bastards. They store the data in ways that you don't expect to find it. Some of the data is very difficult to recognize as anything meaningful. It's not encrypted, but BMW didn't always keep the data together. So many times I had to figure out what the pieces were, and then figure out how to put them back together. Once you put them back together, my CAN analyzer software will let me play back the data and compare it to other signals. Since I've already reverse-engineered the BT Tool protocol, I'm looking for graphs that follow exactly what I captured on the BT Tool CAN bus.

So far I've found these types of data fields:
Clocks and counters. Clocks and counters are those who increase at fixed intervals. They eventually wrap back to the beginning and start all over again. The sizes range from 4-bit to 48-bit counters. Some clocks and counters increase every 5mS, 1-second, and others as long as 10-seconds (and longer). There's a LOT of clocks and counters on the PT-CAN bus.

Analog values. These are digital values that represent analog components. It could be anything from RPM data to motion sensor data.

Binary flags and on/off switches. Every button in the cockpit seems to set a flag in the PT-CAN bus data. These include everything from blinkers to steering wheel switches.

Phase-5: Figuring out exactly what I've found and give them names.
Now that I have the data types figured out, it's time to figure out what they mean. This is also a very difficult task because the data items are often scaled down from larger values. For example, the RPM you see on the tachometer is stored as many-times that value on the PT-CAN bus. Other values are scaled as well, but in non-obvious manners. I've found some items scaled by 5/8 -- kind of an odd scaling factor IMO. In many cases, I know which sensor I've found, but haven't spent the time to figure out the scaling yet.

So far, I've managed to find and decode the following items:

• RPM
• Throttle pedal position
• Brake pedal position
• Steering angle sensor
• Individual wheel speed sensors
• Velocity-1 (possibly GPS speed)
• Velocity-2 (identical to Velocity-1, but seems to have an intentional error built in. This might be the actual speedometer value.)
• Multiple MAF sensors
• Multiple motion sensors (accelerometers, gyro's compass, possibly GPS. About 8-12 sensors in all.)
• High speed "heartbeat" clocks.
• 1-second counter since car started.
• Very rough oil temperature*
• Very rough water temperature*
• Blinkers, headlights,
• Blinker switches, headlight switches, steering wheel switches, etc.
• Many other things I'm probably forgetting to mention.

*Notes:
There's some very notable things that I haven't yet found, and other things I didn't expect to find.

• Gear selection. So far, I haven't found anything to tell me what gear the transmission is in. My tests are exclusively on 6MT. When I run these tests on a DCT, I have a strong suspicion I will find this data.
• Oil and water temperatures. I have found some low resolution versions of oil and water tempurature. Most of the sensor values I've found are high resolution; but the only oil and water temperature sensors I've found have low resolution values. So my search continues for higher resolution values.
• Throttle position. Anybody whose played with the CAN bus files from Racelogic (Video VBox), might have figured out there's a bug in how they handle throttle position. Now that I know where the data comes from, I know exactly what the bug is, and I have fixed it for my own Video vBox files.
• DSC Light. The DSC light doesn't exactly have it's own entry in the PT-CAN bus. Instead, this value must be calculated by looking at various other factors.

Phase-6: Create CAN BUS Database Files (DBC Files):
The CAN analyzer I purchased allows me to create all of these data types of any size and type, then even add scaling to them. The software allows me to export CAN BUS Database Files (DBC files). The Video VBox software can directly import these DBC files. Once imported into the Video VBox software, then the Video vBox can display any of the items in my database onto the screen. This would include the tachometer, steering wheel, gas pedal, brake pedal, blinkers, headlight icons, DSC light, M-Mode buttons, etc.

I'm kind of doing phase-4, 5, and 6 simultaneously. I'm only about 50% done with phase-4, 5, and 6. Hopefully I'll generate some useful information out of this that can be used for anybody with a Video VBox software and tools.

[M3 Sedan JPS]

subbed...

[PencilGeek]

Originally Posted by BrightLight

very nice. All these options are already available on the Vishnu Procede though...Maybe look into that if you want a quicker/easier solution.

Just as I got started on this project, Vishnu and I had two phone calls on this topic, along with some emails between us and their SW developer as well. Vishnu is doing something completely different and accessing the data in a completely different way than what I need. Their approach is similar to what the BT Tool does, and that limits the capabilities and speed of the results (and introduces some very small latencies as well). But more importantly, the data can't be used in the way I wanted to use it.

Not sure what your looking for though

As stated in the article, the goal is to extend the capabilities and virtual gauges on the Video VBox. The Vishu Procede can't do that. That can only be done by finding the data on the PT-CAN bus itself.

[PencilGeek]

Here's a progress update on what I've found so far. There's still plenty of work to do, but so far I have confirmed the following values on the CAN bus. Many of these values are scaled, so figuring out the scaling is another part of the reverse engineering tasks. Unless noted otherwise, I have all of these values figure out, including scaling.

• RPM
• Oil temperature (C)
• Water temperature (C)
• Ambient temperature (needs final confirmation) (C)
• MAF (possible ambient pressure, needs final confirmation) (bar)
• Steering angle sensor (degrees)
• Individual wheel speed sensors (kph)
• Speedometer-Raw Data (kph)
• Speedometer-Adjusted (*Fudge factor shown in cockpit) (kph)
• Individual wheel brake pressure sensors (bar)
• Master brake pedal pressure (bar)
• Brake pedal switch
• Clutch switch
• Gas pedal pressure (%)
• Fuel Gauge
• Odometer (km since driving)
• iDrive settings
• M-Mode settings
• Blinkers
• Headlights
• Steering wheel buttons
• Battery Voltage

And I can see there's many more motion sensors (maybe a dozen more), possibly GPS data, etc. still to figure out.

[imgm3]

Impressive work as always !

[Chebs_E84]

Hey,

Great work on this, I have started to do something similar with my BMW X1 (E84). (To be released later this year in the USA?)
As far as I can see there are a lot of ID's that are the same as the histogram that you posted above, although I am logging the K-CAN rather than the PT-CAN.

So far I have found:-
RPM ,
Coolant temp,
Individual wheel speed,
Time / Date ,
PDC reverse sensor data ,
External Temperature ,
Range,
ODO ,
Steering wheel buttons.
Climate control temperature,
Climate control Fan speed,
Electric window buttons.

I am using a the data to feed into (and control) a computer in the car running Windows-XP with front end software Road Runner, or Ride Runner as it's now named.

I have sent you a PM to see if there is anyway we can help each other.

[NoQuarter]

I'm doing the same work for the E60.

[PencilGeek]

Originally Posted by NoQuarter

I'm doing the same work for the E60.

I suspect E60 and E92 share much of the same data and addresses. PM me and we might be able to share information.

[bigugly]

very impressive work guys

[marcm]

Originally Posted by PencilGeek

I suspect E60 and E92 share much of the same data and addresses. PM me and we might be able to share information.

Hi I too am playing with CAN on the E90 K-BUS and would like to chat with both of you about PIDs. New to the forum so dont think I can PM you yet. Can you please PM me to start the chain.

Thanks
Marc