The Simcenter SCAPTOR product family from Siemens is used for data acquisition with Advanced Driver Assistance Systems (ADAS) and Autonomous Vehicles (AVs).
The Simcenter SCAPTOR MDILink is one part of the
SCAPTOR product family. It is used to help record high-resolution video from cameras utilized in autonomous vehicles and driver assistance systems. The MDI (Measurement Data Interface) Link converts the raw serial data from cameras into Ethernet packets. The device is shown in
Figure 1.
Figure 1: The Simcenter SCAPTOR MDILink for capturing video data and converting to Ethernet based data packets
Using a Simcenter SCAPTOR MDILink has the following advantages;
- By converting the video data to Ethernet packets, the data can be easily recorded with the Simcenter SCAPTOR Recorder or by a user-developed program using provided APIs (Application Programming Interfaces).
- The packets are also precisely time stamped within a microsecond of accuracy. It allows the relative timing of the camera data versus other simultaneously measured signals measured to be well understood.
The Simcenter SCAPTOR MDILink has an onboard application for setting up the device via a web browser.
This article explains the features and functions of the Simcenter SCAPTOR MDILink for collecting raw camera data. It has the following sections:
1. What is the Simcenter SCAPTOR MDILink? What does it do?
1.1 Why?
1.2 Conversion
1.3 Time-Stamping
2. Size
3. Simcenter SCAPTOR MDILink Wiring
3.1 Incoming ports
3.2 Outgoing ports
3.3 MDILink Power
4. Simcenter SCAPTOR MDILink User Interface
4.1 Common Elements
4.2 System
4.3 DTC
4.4 Configuration
4.5 Up-Download
4.6 Console
5. Temperature
6. Additional Software
6.1 MDI-RX API
6.2 rx_sampleapp.exe
6.3 MDI-CMD API
7. Conclusions
1. What is the Simcenter Scaptor MDILink? What does it do?
A Simcenter SCAPTOR MDILink can be used with up to two video cameras at a time. The MDILink deserializes the incoming high-speed data from the camera signal into Ethernet packets. In this process, the data is time stamped.
1.1 Why?
Autonomous vehicles (AV) and ADAS (Advanced Driver Assistance Systems) utilize perception sensors to understand their environment and navigate safely. A combination of radars, cameras, and lidars enables them to get accurately detect obstacles. Cameras and vision systems are key in many of these automated systems, including lane keep assist, adaptive cruise control and automate traffic sign recognition.
Figure 2 shows an example of an automotive and production-grade camera. Nowadays, these cameras are quite compact, which eases their integration all around the vehicle.
Figure 2: Inner workings of a ADAS/AV camera
They all contain an imager board, which hosts the sensor that generates the images. Additionally, an optional ISP (Image Signal Processor) can provide features such as in-camera debayering or automatic exposure control. Finally, the serializer board is critical for transmitting the camera output data over the whole vehicle.
Serializing the output data is needed to transmit it reliably and efficiently to a centralized ADAS ECU (Electronic Control Unit). Indeed, the more conventional CAN-bus and FlexRay do not provide enough bandwidth to transmit the video feeds. Their resolution and frame rate directly affect the performance of the ADAS and are still increasing every year. On the other hand, typical embedded camera protocols like CSI (Camera Serial Interface) are not sufficiently robust against EMC to work over longer distances.
Proven serializing technologies are GMSL2 (Gigabit Serial Multimedia Link) and FPD-Link III (Flat Panel Display Link). They deliver faster data rates while also reducing the cabling costs. They can transmit image and configuration data, as well as power, over the same cable. These serializers also increase the robustness against EMC, making it possible to transmit data over long distances inside the vehicle. However, serializers also imply many challenges when recording cameras, which the SCAPTOR MDILink aims to address.
Read the knowledge article
Camera Recording Tips for Autonomous and Advanced Driving Assistance Systems for more information on recording video with cameras.
1.2 Conversion
The MDILink has two sides, one for managing incoming signals from the camera, and one for outgoing Ethernet based information to a recording device:
- Incoming Signals -Typical camera serial signals come into the MDILink in one of two physical interfaces: Flat Panel Display Link (FPD-Link III) or Gigabit Serial Multimedia Link (GMSL2). The signals are transmitted over a single coaxial cable that includes power for the cameras. Configuration messages between the cameras and the MDILink are also sent over the same coaxial cable using an Inter-Integrated Circuit (I C) channel.
- Outgoing Signals - After converting the incoming camera data into Ethernet data packets, they are sent out on a 10Gb Ethernet connector.
Figure 3 summarizes all data transformations from the imager board up to the recording device. Inside the SCAPTOR MDILink, a matching deserializer receives the data sent by the camera. It passes the raw images to a powerful FPGA (Field Programmable Gate Array) before sending it over to its internal RAM and, finally, to the recorder over 10GbE. One should note that different versions of the SCAPTOR MDILink exist for FPD-Link III and GMSL2, as their camera-facing hardware is slightly different.
Figure 3: From sensor to recorder How is data generated inside a camera and transmitted to a data recording device.
The internal FPGA processes the data on the fly, as shown in
Figure 4.
Figure 4: Workflow describing the conversion of incoming serial data (left) to forwarding of Ethernet packets (right) within the MDILink
The following happens within the MDILink:
- Deserialize: Deserialize the camera data to single line of image (line data)
- Time Stamp: Apply high precision timestamps on line data
- Assemble Images: Make complete image frame from line data
- Cyclic Redundancy Check (CRC): Ensure data integrity
- Packet Transmission: Transmit the data using custom UDP (User Datagram Protocol) packets via a 10Gigabit Ethernet
The Ethernet data uses a custom UDP (User Datagram Protocol) to ensure the data is passed with the appropriate speed. The packets are 9k jumbo frames with a small transport header added. They can be read directly by a Simcenter SCAPTOR recorder. Users can create their program to read the packets using the documentation manual and the provided MDI-RX API which are described further in this document
1.3 Time-Stamping
Accurate timestamps are critical to understanding the relationship between data originating from different sensors. The SCAPTOR MDILink uses hardware timestamping as soon as the data enters its internal FPGA. Therefore, it greatly reduces delays and jitter on the timing data. However, maintaining a consistent time base across the whole recording setup is also required. For instance, data recorded by different MDILinks units should be timestamped based on the same reference clock. A time synchronization mechanism can achieve this.
The MDI can receive 802.1AS time synchronization signals on any of its Ethernet interfaces. These signals can be provided by:
- A SCAPTOR Recorder with XTSS
- An 802.1AS capable switch (1588v2-P2P-UDP or 1588v2-E2E-UDP)
- Directly from an 802.1AS capable grandmaster clock
Consider a large setup with four MDILink units, two recorders, and cameras (simultaneously sending data) as an example use case. The diagram for such a recording system is shown in
Figure 5.
Figure 5: Eight camera, four MDILink, and two recorder system would have less than one microsecond delay
Using the SCAPTOR XTSS, the observed variance on timing data is less than one microsecond and mostly between 100 and 200 nanoseconds. It includes jitter caused by the source and the transmission between source and MDILink. Smaller setups reduce this variance on the timing data even more.
2. Size
The MDILink is approximately 160 mm x 38 mm x 140 mm without connectors as shown in
Figure 6.
Figure 6: Simcenter SCAPTOR MDILink size and dimensions in millimeters
The MDILink has a flange on either side with premade holes for mounting in a vehicle.
3. MDILink Wiring
There are incoming and outgoing physical connections on two sides of the Simcenter SCAPTOR MDILink respectively. Connections include those for the cameras, for connectiions to the recording device, and for power.
3.1 Incoming Ports
One side of the MDILink contains connectors for recording and controlling cameras as shown in
Figure 7.
Figure 7: Simcenter SCAPTOR MDILink connectors for cameras
The camera connectors are:
- Camera1 In: Serial FPD-Link for recording camera video and camera control
- Camera1 Out: In tapping mode, outputs the incoming camera data to not interfere with original camera operation
- Power for Cameras: 4 pin Power-over-Cable (PoC) connector to power cameras
- Camera2 In: Serial FPD-Link for recording video from second camera and for controlling it
- Camera2 Out: In tapping mode, outputs the incoming camera data to not interfere with original camera operation
3.2 Outgoing Ports
The other side of the MDILink contains connectors to output to a recording device as shown in
Figure 8. There are two 10GbE outputs. Each output can carry data from two cameras if needed.
Figure 8: Output ports of the Simcenter SCAPTOR MDILink
Connectors on the SCAPTOR recorder side of the MDILink include:
- Control: 1GB Ethernet connector for configuring the MDILink from the recording device via Ethernet
- Data1 (10GbE 0) Out to Recorder: 10Gb Ethernet connector for output of cameras
- Data2 (10GbE 1) Out to Recorder: 10Gb Ethernet connector for other outputs including high bandwidth camera or mirrored data
- Power for MDI: Five pin connector for powering MDILink, pin five is can be used to turn on and off
The recording device can be a Simcenter SCAPTOR Recorder. Other third party Ethernet recording devices can be used as well. Documentation on the format of the Ethernet packets are provided further in the document, and fully described in the manuals.
3.3 MDILink Power
The power input to power the MDILink is shown in
Figure 9.
Figure 9: MDILink Power Pinout
The MDILink is designed for ignition controlled operation. Connect the ignition signal to Pin 5. It allows a MDILink to be installed in a vehicle and to be turned on and off with the vehicle ignition.
For operations (12 or 24V), please use the power cable included in the delivery contents. Please take care of the correct orientation and diameter for your power cords.
The device must be powered with Safety Extra Low Voltage (SELV-standard EN60950). It is not allowed to connect the device directly to AC line voltages.
The MDILink has a separate power input to power the cameras.
4. Simcenter SCAPTOR MDILink User Interface
The MDILink hosts a web application used to check its status and to configure its settings. This application is accessible while connected to one of its Ethernet ports. The MDILink has a default IP address of 10.0.14.136/255.0.0.0 on the first 10Gigabit Ethernet interface. This IP configuration can be changed anytime but requires a restart to take effect.
The 10GbE interfaces provide access to both configuration and video feeds. However, they only supports 10GBase-T mode, there is no alternative mode available. Accessing the data streamed by the MDILink from a PC or recorder unit requires a 10GBase-T network card. It has to be connected to the first 10GE port of the MDI Link. Alternatively, the 1GbE Ethernet port provides access to the configuration only.
The 10GBase-T network card on the PC or recorder/brick need to be configured as described below:
- IP Address: 10.x.y.x - Everything except 10.0.14.136 is allowed. An IP address of 10.0.14.66 is recommended.
- Jumbo Frames: 9000 - Depending on the driver and OS, this can also be "9k" or "9014" bytes.
- Receive Side: >= 2 Queues
- Receive Buffers: Maximum
- Firewall: Disabled - If the firewall is active, all traffic from the MDILink is blocked. Usually, the firewall is not necessary on this interface because the MDILink normally is the only device connected.
- Browser: Firefox or Google Chrome: It is possible that other browsers cannot display the website correctly or fail to update values.
Access the MDILink setup using the IP address via web browser as shown below in
Figure 10.
Figure 10: Type the IP address into the web browser to connect to the MDILink and configure it
Google Chrome or Firefox is recommended.
If the IP address of the user interface is unknown due to a changed configuration you can use a tool like Wireshark for seeing all incoming packets and thereby getting the proper IP address of the MDI.
When using a MDI Link it is recommended to use the 1 Gigabit management connection, and save the 10 Gigabit connections for camera data.
4.1 Common Elements
After logging into the MDILink via the web browser, the MDILink setup interface is shown (
Figure 11):
Figure 11: Main elements of the MDILink User Interface
All MDI web pages have a common header and footer with the following elements:
- Selection of the subpage: With this you can switch between the different MDI web pages.
- Firmware version & boot mode: Which firmware version is currently installed and from which storage it was booted (either QSPI flash or SD card).
- CPU usage: Shows how busy the MDI CPU is. This shouldn't exceed about 70%.
- Memory usage: Shows how much memory is in use.
- Reboot: Restart the MDILink, for example after changing some settings.
- Open console window: Open a system shell on the MDI for expert debugging.
See Expert diagnostics in the manual for details.
4.2 System
The system page (
Figure 12) shows an overview of the MDILink system status. This can be used to determine if the MDILink is functioning properly.
Figure 12: Sections of the System page of the MDI User Interface
The individual sections of the System page are explained next.
4.2.1 Base System Header
The header area shows basic information about the MDILink unit (
Figure 13).
Figure 13: Base System Header View
Details include all installed versions, the serial numbers, hardware revision, boot mode, and (for the MDILink only) the installed I/O board.
4.2.2 CPU/Memory
Here you can see a history of the MDILink CPU (Central Processing Unit) and memory usage as shown in
Figure 14.
Figure 14: CPU and Memory Usage of the MDILink over time
This shows how busy the MDILink CPU, and normally should not exceed approximately 70%.
4.2.3 Time Sync
This section shows details about the time stamping of the incoming data to the MDILink (
Figure 15).
Figure 15: Time sync status
Normally, the MDILink should be connected to Precision Time Protocol (PTP) source. For example, the
Simcenter SCAPTOR recorder might be providing a time source via IEEE 1588 for time stamping the incoming data. The menu indicates if this happening:
- Time [ns]: The current time in nano seconds (counted from the base time, e.g. 1970-01-01 00:00 for TAI).
- Active: Should be an "x" in this column indicating that the hardware sees the time source.
- Synced: Should be an "x" in this column indicating that time stamping is happening.
- State: Either "slave" or "derived" status.
- Pdelay: Should be close to zero, would indicate a dealy between MDILink and time source.
4.2.4 Network
This area indicates if the MDILink is outputting data on the Ethernet interfaces to the recording device (
Figure 16):
Figure 16: Network traffic status on outgoing Ethernet ports of the MDILink
This is very helpful for debugging. If the MDILink is setup properly, there should be network traffic on the output Ethernet ports.
No traffic might indicate that the MDILink could not resolve target IP address. If the IP address was recently changed, rebooting the MDILink might be needed to fully recognize that the IP address is available.
The network display can also indicate whether a reception problem is on the sending side (MDILink) or on the receiving side (PC/Recorder/BRICK).
4.2.5 Voltage/Temperature
This section shows the voltages and temperatures of the MDILink hardware (
Figure 17):
Figure 17: Voltage and temperature status of MDILink
The temperature display can be quite helpful for checking whether the installation location of the MDI enables sufficient cooling.
4.2.6 Control Application
The control area shows the delay between the time stamping and the actual sending of the data, which should be approximately between 50 and 300 microseconds.
4.3 DTC
The DTC ("Debug Trouble Code") shows detailed logging messages (
Figure 18).
Figure 18: Example of Debug and Trouble Code output of MDILink
Outgoing and incoming signals from the MDILink are monitored. Ideally all messages are green. Message with red indicate potential problems.
Messages are lost when the MDI is turned off.
Please check Basic diagnostics in the manual for more details.
4.4 Configuration
After configuring a MDILink, the configuration can be stored in a file. This file could be used to configure other MDILink units that might be using the same camera.
The MDILink configurations are stored in json files. Those files reside on the device and can be modified via the WebGUI. Enter the IP of the device into the address bar of your web browser and select the "config" tab in the MDILink User interface:
- There will be at least one configuration file present. The WebGUI can be used for editing the config files, all input is validated automatically.
- Uploading: Configuration files can be uploaded (next section). When uploading the configuration it is checked for compatibility. If compatibility problems are identified, the config might be discarded and replaced by a default one, or the MDI will notify about an issue in the DTC log in the system tab.
For information about the JSON specification, see http://www.json.org
4.5 Up-Download
In the up-download interface, a previously created json file for the old camera can be uploaded to the MDILink as shown below (
Figure 19).
Figure 19: The up-download interface from the MDILink User Interface
A common use would be to setup a new MDILink with a previously used camera.
During the upload, the json file will be double checked to make sure it is a valid format. The device checks the configuration against a schema. If for some reason the configuration won't pass the scheme check, a default configuration will be generated automatically. It is advised to back up the configuration regularly, especially prior to any system update.
4.6 Console
You can open the console at the bottom of each page with the arrow button. This console is useful for expert debugging.
5. Temperature
MDILink temperature information:
- Allowable operating temperatures for the MDILink can range from -20 degrees Celsius to +70 degrees Celsius.
- Storage temperatures can range from -40 degrees Celsius to 85 degrees Celsius.
6. Additional Software
Additional software that is provided with the Simcenter SCAPTOR MDILink allows it to be interfaced with user written applications.
6.1 MDI-RX API
MDI-RX API is a generic high-performance reception API (Application Programmable Interface) for all MDILink devices, both for Windows and Linux.
The API does the following:
- Collection and combination of the individual UDP packets sent by the MDILink to complete frames returned to the API user.
- If UDP packets are lost, the frame is either discarded or marked as corrupt depending on the API configuration.
- It only delivers the full MDI frames without interpretation of the header or extraction of the data. Both are up to the application that uses this API.
- Return of reception statistics, e.g. the number of received frames, the quantity of memory used, etc.
Be aware that other devices in this setup like switches also must support IP Multicast to make it work correctly.
6.2 rx_sampleapp.exe
The MDI-RX API contains the rx_sampleapp.exe that supplies basic debug functionality:
- Serves as a minimum reception example to see if incoming data is clean.
- Dumps the received MDI Frames by pressing the key 'd'. Send this dump to Siemens support for further analysis.
6.3 MDI-CMD API
The MDI-CMD API is a device dependent API for control and interaction for Windows and Linux. It consists of two parts:
- A common part that shows general MDILink information like type and serial number.
- A hardware dependent section that enables some interface specific actions like sending I2C commands.
More information about the APIs are provided in the MDILink user manual.
7. Conclusion
The Simcenter SCAPTOR MDILink hardware enables flexible access to the raw sensor data for the development of ADAS/AV perception algorithms, as well as for the validation of new sensor technologies.
Hope this article helps you with using the Simcenter SCAPTOR MDILink!
Questions? Contact
charles.rice@siemens.com or the
Siemens Support Center.
Related Links: