BitSim is developing a system for geophysical exploration in boreholes using a chain of hydrophones and digital communication units. This is done in cooperation with the Smart Exploration EU funded project. This video is from a recent field test. Anton Lindström from BitSim and Christopher Juhlin from Uppsala University are explaining how the system works.
BitSim has developed a receiver for FLIR’s Video over SPI (VoSPI), an interface to enable streaming images from a Lepton Infrared camera directly to an FPGA-based image processing system. You can use it in your platforms like:
- On Xilinx devices with our new customized IP.
- On every SoC circuit with an ARM CPU and Python with our pure-software driver.
- A Python interface which integrates the VoSPI IP in your PYNQ design.
VoSPI stands for “Video over Serial Peripheral Interface”. VoSPI protocol is designed to send out the video in a format that allows transmission over a SPI interface while requiring minimal software or hardware. The sensor acts as SPI slave and the hardware acts as SPI master and the video is streamed on MISO pin. The hardware system uses custom logic to receive and render the video. The sensor sends out bytes of pixels through packets and segments to form a frame of 160×120 resolution.
The development of this IP has been done on BitSim’s Python-based development platform, SpiderPig board. Utilizing this simple interface between the Logic fabric and the high-level Python environment, debug information and image analysis could be performed almost directly after a bitfile is generated. BitSim has developed tools for Thermal Imaging and specifically to integrate the FLIR Lepton sensor by using VoSPI.
Using this IP block, it is possible to attach a low cost FLIR Lepton IR sensor, which sends processed 16-bit data to an FPGA design. The IR sensor captures infrared radiation as input. The output is a uniform thermal image with temperature measurements throughout the image. This can be used in applications such as Mobile phones, Gesture recognition, Building automation, Thermal imaging and Night vision where detection of temperature values and high temperature scenes are necessary.
BitSim develops electronics for product companies, focusing on Imaging and Edge Computing. We see a constant influx of new sensors, interfaces and key components. With the following few words we want to tell you what we think is interesting in the market, but also bring up experiences, difficulties and things to think about. And, of course we would be happy to discuss your specific needs and solutions.
It can be really difficult to get sensors running with all configurations needed. Sometimes we find features that are not even documented. And the sensors often have hundreds of registers where most of them have to be configured in the correct way to get an image.
- BitSim has developed a camera with Sony’s IMX290-sensor that has very good lighting properties, i.e. can handle difficult lighting conditions. It has 10/12-bit ADC, MIPI interface, resolution up to 1080p, up to 120 fps. Flipped sideways, the resolution becomes 1109×1945 pixels. There are also a couple of HDR variants available to enable further light enhancing functionalities.
- FLIR’s Lepton is a relatively inexpensive IR sensor that can be used separately, or in conjunction with standard CMOS sensors to extract additional information from the image through so-called “image fusion”.
- CCS. We at BitSim hope that sensor or module suppliers will adopt MIPI’s initiative CCS – Camera Command Set: https://mipi.org/specifications/camera-command-set. The idea is to quickly get started with a sensor with its basic functionality without specific SW drivers. A typical command set can be handle things like resolution, frame rate and exposure time, but also more advanced features such as autofocus and single or multiple HDR.
This is often a factor making things in the development project more complex with extremely small connectors that easily break, or become loose with a bad connection.
- We have developed a dozen different sensor adapter cards that fit development cards from Xilinx, NXP and Technexion etc. for rapid prototyping. It is a lot to think about before these small adapter cards work well, as there are usually different types of cables, connectors and sizes needed.
- 4K Video BitSim has implemented 4K @ 60 Video, i.e. HDMI from an FPGA. In this project, we divided the camera into two physically separated parts, Front end (Camera sensor) and Back end (processing unit) with Aurora in between, i.e. Xilinx high-speed series protocol.
- MIPI CSI-2 We have continued the development of our own camera interface IP, which now supports FPGAs with built-in D-PHY IOs (which has the advantage that no external resistance networks or Meticom circuitry are needed), e.g. a Xilinx UltraScale+ / MPSoC. Now you can get 2.5Gb / s per lane!Processing (Platforms & Algorithms)
One alternative for the processing of the image chain is a combined CPU and FPGA circuit, e.g. Zynq / MPSoC with the possibility to process in C / C ++ and VHDL.
- We have worked with Python, C / C++ and the open image library Open CV to adapt the contents of an image. With Xilinx Vision (HLS Video Library), it is also possible to use hardware accelerated OpenCV calls.
- Another alternative is to process in a SoC circuit, i.e. with an ARM CPU, software and built-in fixed accelerators. NXP (formerly Freescale) has had great success with the i.MX6 family. The next generation, i.MX8, has been available for a couple of years. We have been working with the i.MX8 for a little more a year, and we now experience that NXP’s libraries, documentation and forums are starting to become really useful.
- We have a complete video chain, i.e. from glass-to-glass (sensor to screen), via MIPI CSI-2, V4L and Gstreamer with H.264 compression, via Ethernet to the screen.
Please contact us if you are interested or have questions!