IR Sensor Linux – V4L SW Driver (VoSPI)

BitSim NOW has a V4L (Video4Linux) SW driver available for VoSPI.

FLIR Lepton IR sensors use a dedicated format called VoSPI, Video over SPI. MIPI CSI-2 is a standard video interface format commonly used by image sensors manufacturers.
BitSim NOW has developed an IP block – Bit-VoSPI MIPI – that bridges the two standards.

The hardware IP block by BitSim NOW, implemented in an FPGA, converts VoSPI to MIPI CSI-2.

BitSim NOW has implemented Linux support for Bit-VoSPI MIPI and we have a working demo setup.

The Software is built on Yocto – Linux + Video for Linux together with gstreamer. A histogram filter function is added to amplify the details of the IR image. And finally, a dedicated gstreamer client application has been created for the receiving PC side to handle monochrome video.

The demonstration system consists of the sensor, an FPGA card and an i.MX8 ARM board running Linux.

The BitSim NOW VoSPI FPGA IP block has been tested on Xilinx and Microchip platforms.

The BitSim NOW MIPI-CSI2 block is available for FPGAs from Xilinx, Intel and Microchip.

To read more about the BitSim VoSPI IP: https://bitsimnow.se/bitvospirx-for-flir-lepton-ir-sensor/

To read more about MIPI CSI-2: https://www.mipi.org/specifications/csi-2

See more at https://bitsimnow.com/ip-block/ or contact info@bitsimnow.com

Master Thesis – Sports and AI

Improving distance and impact location measurements in athletic sports using state-of-the-art machine learning/machine vision.

This thesis project is an excellent way to get hands-on experience using AI to solve complex computer vision problems and learn how to go from a system design to a hardware prototype. This thesis-project aims to design and build a prototype of a camera-tracking system that can determine where objects land within a fixed area. Although similar systems already exist today, the novelty of this project lies in the fact that we are trying to increase the area that can be monitored.

Background: Did you win the Olympics, or did you “only” finish fourth?

This question is a reality for many athletes in today’s world, especially in events such as javelin throw, discus throw, hammer throw and, shot put. To measure where the object lands is a manual task conducted by humans; therefore, mistakes and errors in the measurement process are not uncommon. The person making the measurements must avoid being hit by a deadly projectile, which does not make the task any easier. Imagine training your entire life and not getting that well-deserved medal because of a measurement error. To prevent this from happening, we want to try and design a computer-vision system that can assist in the measurement process for the events mentioned above. Since tracking flying objects is a general task, a functional prototype could be of interest in many other areas and not just sports events. You do not need to be interested in these events or have any previous knowledge of the events to complete this project.

Javelin

Problem formulation: Can we use modern and intelligent technology (AI) to measure the length of the throws of one or several of the following events: javelin throw, discus throw, hammer throw, and shot put, or should it only be used as a complement to the current measurement process.

Target: To construct a prototype using a set of cameras connected to a computer that can measure the length of the throws, we want the system to present a result in under 10 seconds from the moment of impact.

  • Below is a compiled list of some of the questions/problems that need to be addressed in this project.
  • How do different surfaces .e.g grass, gravel, plastic, affect the system?
  • How do we make sure that the system only records the first impact (objects can bounce)?
  • How to make correct measurements when the object lands abnormally, .e.g a javelin that lands flat on the ground?
  • How many camera sensors are needed, and where should they be placed relative to the thrower?
  • Which type of camera sensors are best suited for this task?
  • Which AI model should be used to achieve a good mix of accuracy and performance?
  • How do we collect and store data that the model can be trained on?

Required background: Image processing, AI/ML, CNN, OpenCV, Python, or C++.
Start date: January 2022. End date: June 2022.
Contact: Niclas Jansson – BitSim Now

BitSim and NOW Electronics joins forces

The embedded system developers BitSim and NOW Electronics, with offices in Stockholm and Växjö, will merge their operation to form BitSim NOW with 40 employees and 15 sub-contractors.

NOW Electronics, was established 1985 and BitSim 2000. Both companies work with electronics development, sensor technology, embedded computing, machine vision machine learning and accelerated Imaging.

“Our companies complement each other in terms of market position and competence, where we will get a substantial increase in the FPGA area,” says Philip Nyströmer, CEO of NOW Electronics and the new merged company BitSim NOW. “Together, we are now equipped to take on larger and broader assignments in our new premises”.

“We are looking forward to working with NOW Electronics’ talented developers and thereby increasing our activities in image processing and sensor technology” says Anders Sivard, CEO of BitSim AB.

Electronics is becoming strategic in more and more markets. Through this merger, it will be possible to meet a larger demand for built-in, sensor-centered, and interconnected electronics systems, increasingly important for the whole industry.

 

For more information, contact Philip Nyströmer, +46-72-0798523

Polarfire FPGA and MIPI CSI-2 IP

BitSim’s Camera Interface IP, MIPI CSI-2, now supports FPGAs from Microsemi in the PolarFire series. Both MIPI-CSI2 for PolarFire FPGAs without processor, at chip footprints as small as 11×11 mm, and also MIPI-CSI2 for the new PolarFire SoC with built-in RISC-V processors.

For more information, contact BitSim at info@bitsim.com

Polar MIPI
BitSim MIPI CSI-2 Tx on PolarFire, connected to an i.MX8 SOM (Varscite)

Adapter Cards and Interfaces

BitSim has initiated an open connector standard for camera modules: OMIPICON. OMIPICON stands for Open MIPI CONnect and is suitable for prototypes or production of small/medium-sized quantities.

The idea behind this is to save time and money when developing hardware with camera sensors. Neither the MIPI CSI-2 standard nor the MIPI DSI standard define a specific connector which means that suppliers of sensor modules use their own connectors, incompatible with others. You then need custom designs.

In addition, most available sensor connectors today are not suited  for repeated inserts and removals. When debugging prototypes with these sensors, quite often these connector are only capable to withstand a few connections and disconnections. You end up spending too much time on connector issues.

With OMIPICON, there is only need for one FMC-adapter board and one U96 adapter board. And one adapter board per sensor. You then don’t need to insert and remove the adapter board’s connector.

adapter card camera modules
adapter card for contacting camera modules

camera moduel

 

Testing our latest product

Usually we at BitSim help create things that are physically small, like PCBs and FPGA configuration. If we have have to use a ruler longer than 10cm we consider something to be ”large”. Not any more. Last October we performed a validation test of our latest product, and that is quite a bit bigger.

The product allows synchronous measurements to be taken over a long distance. All sensors, in this case hydrophones, can be daisy-chained. For the prototype we had a cable of 500m between the controlling electronics and the first “node” that takes the measurements from the hydrophones and sends the data to the controlling node. After that we had 20 meters between the remaining nodes. Well that was the idea, we still needed to prove that it works.

To to that we, together with partners from the Uppsala University, went to test the system. Remember that we are used to small products? This one was so big, it needed to be transported on a trailer. Since we had assembled the system in Uppsala we went there to help packing and loading the trailer. After reading the manual of the trailer a few times and some trial-and-error we got the system on the trailer and were ready to go.

Car in front of the trailer with the downhole measurement system
Everything is ready for the long drive to Ludvika

Continue reading “Testing our latest product”

Geophysical exploration in boreholes

BitSim is developing a system for geophysical exploration in boreholes using a chain of hydrophones and digital communication units. This is done in cooperation with the Smart Exploration EU funded project. This video is from a recent field test. Anton Lindström from BitSim and Christopher Juhlin from Uppsala University are explaining how the system works.

IR sensor interface

BitSim has developed a receiver for FLIR’s Video over SPI (VoSPI), an interface to enable streaming images from a Lepton Infrared camera directly to an FPGA-based image processing system. You can use it in your platforms like:

  • On Xilinx devices with our new customized IP.
  • On every SoC circuit with an ARM CPU and Python with our pure-software driver.
  • A Python interface which integrates the VoSPI IP in your PYNQ design.

VoSPI stands for “Video over Serial Peripheral Interface”. VoSPI protocol is designed to send out the video in a format that allows transmission over a SPI interface while requiring minimal software or hardware. The sensor acts as SPI slave and the hardware acts as SPI master and the video is streamed on MISO pin. The hardware system uses custom logic to receive and render the video. The sensor sends out bytes of pixels through packets and segments to form a frame of 160×120 resolution.

The development of this IP has been done on BitSim’s Python-based development platform, SpiderPig board. Utilizing this simple interface between the Logic fabric and the high-level Python environment, debug information and image analysis could be performed almost directly after a bitfile is generated. BitSim has developed tools for Thermal Imaging and specifically to integrate the FLIR Lepton sensor by using VoSPI.

Using this IP block, it is possible to attach a low cost FLIR Lepton IR sensor, which sends processed 16-bit data to an FPGA design. The IR sensor captures infrared radiation as input. The output is a uniform thermal image with temperature measurements throughout the image. This can be used in applications such as Mobile phones, Gesture recognition, Building automation, Thermal imaging and Night vision where detection of temperature values and high temperature scenes are necessary.

IR video over SPI
Thermal image of  a person holding a hot coffee captured by Lepton 3.5 IR sensor.