IR Sensor Linux – V4L SW Driver (VoSPI)

BitSim NOW has a V4L (Video4Linux) SW driver available for VoSPI.

FLIR Lepton IR sensors use a dedicated format called VoSPI, Video over SPI. MIPI CSI-2 is a standard video interface format commonly used by image sensors manufacturers.
BitSim NOW has developed an IP block – Bit-VoSPI MIPI – that bridges the two standards.

The hardware IP block by BitSim NOW, implemented in an FPGA, converts VoSPI to MIPI CSI-2.

BitSim NOW has implemented Linux support for Bit-VoSPI MIPI and we have a working demo setup.

The Software is built on Yocto – Linux + Video for Linux together with gstreamer. A histogram filter function is added to amplify the details of the IR image. And finally, a dedicated gstreamer client application has been created for the receiving PC side to handle monochrome video.

The demonstration system consists of the sensor, an FPGA card and an i.MX8 ARM board running Linux.

The BitSim NOW VoSPI FPGA IP block has been tested on Xilinx and Microchip platforms.

The BitSim NOW MIPI-CSI2 block is available for FPGAs from Xilinx, Intel and Microchip.

To read more about the BitSim VoSPI IP: https://bitsimnow.se/bitvospirx-for-flir-lepton-ir-sensor/

To read more about MIPI CSI-2: https://www.mipi.org/specifications/csi-2

See more at https://bitsimnow.com/ip-block/ or contact info@bitsimnow.com

Polarfire FPGA and MIPI CSI-2 IP

BitSim’s Camera Interface IP, MIPI CSI-2, now supports FPGAs from Microsemi in the PolarFire series. Both MIPI-CSI2 for PolarFire FPGAs without processor, at chip footprints as small as 11×11 mm, and also MIPI-CSI2 for the new PolarFire SoC with built-in RISC-V processors.

For more information, contact BitSim at info@bitsim.com

Polar MIPI
BitSim MIPI CSI-2 Tx on PolarFire, connected to an i.MX8 SOM (Varscite)

Adapter Cards and Interfaces

BitSim has initiated an open connector standard for camera modules: OMIPICON. OMIPICON stands for Open MIPI CONnect and is suitable for prototypes or production of small/medium-sized quantities.

The idea behind this is to save time and money when developing hardware with camera sensors. Neither the MIPI CSI-2 standard nor the MIPI DSI standard define a specific connector which means that suppliers of sensor modules use their own connectors, incompatible with others. You then need custom designs.

In addition, most available sensor connectors today are not suited  for repeated inserts and removals. When debugging prototypes with these sensors, quite often these connector are only capable to withstand a few connections and disconnections. You end up spending too much time on connector issues.

With OMIPICON, there is only need for one FMC-adapter board and one U96 adapter board. And one adapter board per sensor. You then don’t need to insert and remove the adapter board’s connector.

adapter card camera modules
adapter card for contacting camera modules

camera moduel

 

12 HD camera sensors streamed 14 Gigabit/s to a PC

The design consists of 2 cards, each with an FPGA. Each FPGA receives 6 1280 x 800 HD camera sensors 120 frames per second.

Each FPGA streams the 6 channels to a 10Gb IP UDP Ethernet block (Our own IP block) directly to a PC. Everything is done in pure HW, none of the video flow is handled by the ARM CPU in the PGA in this version. Each 10Gb Ethernet cable transfers 70% of full HW speed, i.e. 7 Gbps, at a total of 14 Gbps for the PC to receive and render.

Of course, FPGAs can also encode and compress incoming data to reduce image flow or process early.

6 Sensors
12 HD camera sensor streamed 14 Gigabit/s to PC

IR sensor interface

BitSim has developed a receiver for FLIR’s Video over SPI (VoSPI), an interface to enable streaming images from a Lepton Infrared camera directly to an FPGA-based image processing system. You can use it in your platforms like:

  • On Xilinx devices with our new customized IP.
  • On every SoC circuit with an ARM CPU and Python with our pure-software driver.
  • A Python interface which integrates the VoSPI IP in your PYNQ design.

VoSPI stands for “Video over Serial Peripheral Interface”. VoSPI protocol is designed to send out the video in a format that allows transmission over a SPI interface while requiring minimal software or hardware. The sensor acts as SPI slave and the hardware acts as SPI master and the video is streamed on MISO pin. The hardware system uses custom logic to receive and render the video. The sensor sends out bytes of pixels through packets and segments to form a frame of 160×120 resolution.

The development of this IP has been done on BitSim’s Python-based development platform, SpiderPig board. Utilizing this simple interface between the Logic fabric and the high-level Python environment, debug information and image analysis could be performed almost directly after a bitfile is generated. BitSim has developed tools for Thermal Imaging and specifically to integrate the FLIR Lepton sensor by using VoSPI.

Using this IP block, it is possible to attach a low cost FLIR Lepton IR sensor, which sends processed 16-bit data to an FPGA design. The IR sensor captures infrared radiation as input. The output is a uniform thermal image with temperature measurements throughout the image. This can be used in applications such as Mobile phones, Gesture recognition, Building automation, Thermal imaging and Night vision where detection of temperature values and high temperature scenes are necessary.

IR video over SPI
Thermal image of  a person holding a hot coffee captured by Lepton 3.5 IR sensor.

Bit-UDP Ethernet -1G IP/UDP full HW stack transmitter / receiver

Ethernet to transfer large amount of data
To efficiently transfer large amount of data and off-load the embedded processor, BitSim is offering its IP core, Bit-1G-UDP Ethernet.

This core offloads an embedded processor with the communication accelerated in the FPGA, with more than ~900+ Mb/s of effective data transfer rate. Standard functions for transmit, receive, ARP-handling, AXI-stream and loop back are included.