Active and Passive Air Recuperation

Low level System, Energetics, C, Control and Sensoring

Domestic heating has always been problematic. Indoor climate control had to be established to maintain comfort and prevent the spread of disease. In the late 20th Century, we began to ask ourselves about energy efficiency. What was the cost of maintaining this comfort and health? Instead of finding the one solution to solve our energy needs, what about if we looked closer, and at the energy we already have?

Then I thought, “what about Energy Recuperation?”

I constructed a unique, thermoelectric heat pump that was able to recycle the energy in the warm air escaping from a building into fresh air being drawn in. The device consisted of passive and active components, controlled by an embedded system built upon the Arduino platform. I also developed the connectivity aspects and successfully deployed the system.

While in high-school, I won a gold medal at The International Sustainable World Energy, Engineering, and Environment Project Olympiad (I-SWEEEP) in Houston, Texas with this project. I also applied for, and was awarded a national patent for the device. This was the first time I was working with embedded devices- completely individually. In addition to all the technical skills that I acquired through building multiple prototypes, I learned a lot about project planning and how to present my ideas to a wider audience.

• Date: 2014-2016

FPGA Marker Detection

FPGAs, Computer Vision, C++, High-Level Synthesis

Novel ways to interact with computer systems, especially through motion capture and tracking, is becoming increasingly important. From information navigation to game play and even authentication, it is now more than just an idea from a James Bond movie. To continue to pursue more natural interaction, there is now a real need to develop dedicated, power efficient, embedded hardware that is streamlined for the task at hand.

This strive towards smarter ways to interact with a computer system formed the basis of my first-year project at Imperial College London. Alongside Alexander T. Luisi and George Punter, we developed a hardware accelerated background-subtraction algorithm, with statistical point prediction to track and act upon a user’s motion. We also developed and demonstrated a marker-based approach to highlight alternative methods to track motion using Field-Programmable Gate Arrays (FPGAs).

This project was an excellent showcase of the power of FPGAs for real-world applications. I gained a great amount of insight into High-Level Synthesis (HLS) tools as well as the design of complex digital circuitry. I also learned about Digital-to-Analogue Converters (DACs) and Analogue-to-Digital Converters (ADCs) to be able to simultaneously control music according to the the data received from a camera module.

• Date: June 2016

ZSD Toolbox

Java coding, Optimizations, Consultancy, Presenting

Západoslovenská Distribučná a.s. (ZSD) is the major electricity distribution company in the western part of Slovakia. More than one million households depend on ZSD for their electricity, and so to ensure that everyone stays connected to the grid, ZSD collects data by means of Smart Meters, deployed throughout western Slovakia. Data collection is nothing without data analytics and processing, and this needs to be done for each and every customer.

These smart meters lacked support in the ZSD’s software stack, meaning that swathes of customer data had to be stored and processed manually, costing time and money. Upon my entry to the company as an intern, I was tasked to develop a software toolbox in Java to process and format this data so that it could be imported into an existing database.

The project was a success, and is still relied upon today- a great achievement considering that this was my first internship in a corporate environment. I learned a great deal about how to interact with other software developers and the platform’s target user base, as well as how a department functions within the wider company. ZSD’s CEO Andrej Juris summary of my internship best describes the impact of my work, “it will help our company in many ways to streamline work processes and thus contribute to significant savings in cost and time.”

• Date: July-September 2016

CPU Simulator

C++, Assembly, Compilers & Simulators

One part of the EIE second year Computer Architecture course was to develop, in C++, a CPU Simulator, based on the MIPS1 chip architecture. This virtual CPU was built in tandem with an automatic verification platform designed to exercise its capabilities. The deliverable had emphasis on rigorous memory management, data format encoding, including endianness as well as simulating exceptions and branch handling. The CPU and the verification platform followed a strictly defined Application Programming Interface (API) to make them portable and sustainable for potential future uses. It also enabled for my colleagues and me to test our work against each other’s frameworks, to ensure that our solutions were as rigorous as possible.

Reflecting on this coursework, it was a tremendously useful exercise that taught me in greater detail about memory management in C++ and improved my general programming skills. It also solidified my understanding of basic processor architectures, particularly Reduced Instruction Set Architectures. You can try my work for yourself at: Link.

Later in the course, I developed a compiler that took C89 code and compiled it down to the MIPS1 Instruction set. The output of this compiler would also be simulated using QEMU but could be ported to the CPU simulator. Once again, this compiler can be found at: Link.

• Date: 2016-2017

Concussion Prevention

Embedded System, Healthcare, Python, Embedded C

With concussions being reported as one of the most frequent injuries in Ice Hockey, and renewed efforts to better understand Traumatic Brain Injuries (TBI), there is demand to enhance identification of a potential injury. Studies have shown that NHL players experience an average of 1.8 concussions every 1000 hours of play and almost double that for juniors, with 12% reporting a head injury. Of these injuries sustained, over half may go unreported, demonstrating that concussions are a regular occurrence across all leagues.

Working with Owen Harcombe, Alexander T. Luisi and Alexander Montgomerie-Corcoran, we developed a prototype Internet of Things product, that sits inside a player’s helmet to detect and track impacts while they are on the ice. The embedded device incorporates a gyro and accelerometer to measure these forces, and uses a WiFi enabled microcontroller to report the data back to a central server. The data is then processed and delivered to the user via an intuitive web application, making it accessible across mobile and desktop. We employed machine learning to better interpret the sensor data, and I added a unique touch by using a K-means algorithm to filter the impacts reported by the sensors, alleviating the need for long sensor calibration.

Another part of the coursework was to develop a marketing and business strategy for the prototype product. Having always been on the engineering side of product development, this was an entirely new experience, and one that I found very rewarding, as well as a skill that I would like to develop further. You can see our concept demonstration here:

• Date: March 2018

Frisch

Embedded System, Machine Learning, Food Waste, Python

The United Nations Food and Agriculture Organization (UNFAO) estimates that one third of food worldwide ends up being wasted. In real world terms, this is around 1.3 billion tons a year. In fact, UK households alone, waste on average almost £13 billion a year, which is a staggering figure.

I formed a team with George Punter, a fellow undergraduate at Imperial along with Slovak Technical University undergraduates Patrik Štefka and Filip Hatiar, to combat this food waste epidemic, starting at the root cause of the problem in the developed world. The refrigerator. Our concept combines smart, sensor based hardware and intelligent machine learning software to recognize the food in the fridge, and when it might expire, reminding users that they should make use of certain ingredients if they are to avoid wasting their food purchases. It also makes handy recipe suggestions using what is currently in the refrigerator.

The biggest challenge that we had to overcome was integrating all of this sensor technology with the machine learning algorithms on an embedded platform, such as the Raspberry Pi. Through clever programming and neat design choices, we managed it, and are now building our first prototype. More detail can be found at: Link.

• Date: 2017-2018

FPGA/AI Toolbox

FPGAs, C++, Machine Learning, Python

Machine Learning has skyrocketed to become a key focal point for researchers in all fields, with industry quickly adopting the technology to improve their products, services and internal processes. However, while the frameworks have become more accessible through companies such as IBM and Google, deploying and training ML solutions is still laborious, time consuming and difficult to resource. Resources use is where FPGAs come in.

Reconfigurable hardware is an ideal platform as it allows for algorithms to have dedicated hardware, without the need to develop physical silicon in the form of an ASIC. Unfortunately, FPGAs also suffer from pitfalls. The main issue is the overhead and knowledge required to configure them using Hardware Definition Languages such as Verilog.

Due to my previous experience with FPGAs, I joined Corerain Technologies Ltd, based in London and Shenzhen to embark on my six-month industrial placement. The company recognized that omitting the high level synthesis stage removes the knowledge requirements and development overheads associated with FPGA application development. My role was to develop this concept, by using templates extracted using Machine Learning frameworks. These templates are then translated into Verilog, which act as parameterized building blocks, which can be assembled into a data-flow graph to form ML programs. The graph is the compiled onto an FPGA, ready to receive data and pass it through the algorithms. These blocks are encapsulated into a toolchain that enables people to make use of accelerated ML technology but do not have experience with FPGAs.

Working at Corerain was a unique and rewarding experience. Being a midsize and international startup, I learned how to work in a highly dynamic environment while also manage my responsibilities over a number of timezones. It was fascinating to carry out research work while working as an engineer.

• Date: April-September 2018

F-E3D: FPGA-based Acceleration of an Efficient 3D Convolutional Neural Network for Human Action Recognition (Best Paper Candidate)

2019 International Conference on Application-specific Systems, Architectures and Processors (ASAP)

H. Fan, C. Luo, C. Zheng, M. Ferianc, Z. Que, S. Liu, X. Niu, W. Luk

Three-dimensional convolutional neural networks (3D CNNs) have demonstrated their outstanding classification accuracy for human action recognition (HAR). However, the large number of computations and parameters in 3D CNNs limits their deployability in real-life applications. To address this challenge, this paper adopts an algorithm-hardware co-design method by proposing an efficient 3D CNN building unit called 3D-1 bottleneck residual block (3D-1 BRB) at the algorithm level, and a corresponding FPGA-based hardware architecture called F-E3D at the hardware level. Based on 3D-1 BRB, a novel 3D CNN model called E3DNet is developed, which achieves nearly 37 times reduction in model size and 5% improvement in accuracy compared to standard 3D CNNs on the UCF101 dataset. Together with several hardware optimizations, including 3D fused BRB, online blocking and kernel reuse, the proposed F-E3D is nearly 13 times faster than a previous FPGA design for 3D CNNs, with performance and accuracy comparable to other state-of-the-art 3D CNN models on GPU platforms while requiring only 7% of their energy consumption.

A Real-Time Object Detection Accelerator with Compressed SSDLite on FPGA (Best Paper Candidate)

2018 International Conference on Field-Programmable Technology (FPT)

H. Fan, S. Liu, M. Ferianc, HC. Ng, Z. Que, S. Liu, X. Niu, W. Luk

Convolutional neural network (CNN)-based object detection has been widely employed in various applications such as autonomous driving and intelligent video surveillance. However, the computational complexity of conventional convolution hinders its application in embedded systems. Recently, a mobile-friendly CNN model SSDLite-MobileNetV2 (SSDLiteM2) has been proposed for object detection. This model consists of a novel layer called bottleneck residual block (BRB). Although SSDLiteM2 contains far fewer parameters and computations than conventional CNN models, its performance on embedded devices still cannot meet the requirements of real-time processing. This paper proposes a novel FPGA-based architecture for SSDLiteM2 in combination with hardware optimizations including fused BRB, processing element (PE) sharing and load-balanced channel pruning. Moreover, a novel quantization scheme called partial quantization has been developed, which partially quantizes SSDLiteM2 to 8 bits with only 1.8% accuracy loss. Experiments show that the proposed design on a Xilinx ZC706 device can achieve up to 65 frames per second with 20.3 mean average precision on the COCO dataset.