• What Is It? – Simulated Annealing program for studying crystal defect formation on 2D manifolds
• Technologies Used – Julia
• My Accomplishments
  • Improved algorithm speed by 40x
  • Drove project to completion
  • Effectively communicated technical requirements with a non-technical stakeholder

This project is part of a team effort to study colloids movement on 2D manifolds, the formation of lattice defects, and how they relate to the morphing of viral capsids. The team was led by Dr. Luigi Perotti at the University of Central Florida.

What it Does

This program simulates particle interaction using a modified version of the Kinetic Monte Carlo Algorithm. After taking in a number of parameters including the number of particles, the number of iterations, and the equilibrium distance, the program initializes a set of randomly positioned particles.

During each timestep, each particle is perturbed and this move is either saved or reverted based on whether it lowers the total system energy. Eventually, the lowest possible energy configuration is reached.

How I Did It

I started with the pseudo-code for the modified Kinetic Monte Carlo algorithm. Using the Julia programing language, I implemented the algorithm along with supporting features such as estimated completion time, outputting an energy-time graph, and creating a particle movement animation. This iteration was extremely slow, but it worked.

Improvements I Made

I was able to double the speed of my program simply by reducing repeated computations. For example, if I calculate the distance between particle a and particle b, then I don’t need to compare the distance again between b and a. I was also able to utilize parallelization, as each particle movement is independent.

From here I completely restructured the entire program to improve the energy calculation time complexity from O(n²) to O(n). This was a huge improvement because it increase the maximum number of particles from around 30 to a couple thousand. I did this by spatially encoding the position of each particles in memory. I broke the region up into zones and each zone was an independent particle array. Each particle only needs to check it’s own zone, and the eight adjacent zones. Because zones scale with equilibrium distance, each particle only needs to compare to a small constant number of other particles.

Overall, the improvements I made were able to speed up the original program by over 40 times.

What I Learned

This was the most challenging project I have ever completed and I learned so much during the process. First, I had never worked with this programming language before, but now I would consider myself proficient in Julia. Another thing I learned was how to obtain technical requirements from a non-technical person. I was given the task to “implement this algorithm” and had to figure everything out on my own. This taught me how to ask questions and learn what the goal of the project was, as well as determine what features would contribute to the goal. In implementing parallelism and my spatial encoding scheme, I learned a lot about memory management and working with shared memory. The final big lesson was learning how to develop software as a piece of a bigger project. My program need to accept inputs from a completely separate program and then map outputs in a way the next program would understand. Communicating with my team and being able to work together to line everything up was definitely challenging, but I developed a deeper understanding of team dynamics in the process.

• What Is It? – Autonomous Drone for last-mile package delivery
• Technologies Used – C++, Python, Raspberry Pi, PX4, Linux, ORBSlam 3.0
• My Accomplishments
  • Decreased system unit cost by 40%
  • Implemented many different software into a cohesive system
  • Gathered technical requirements and design specifications
  • Designed and actualized project from start to finish

Background

Last mile deliveries (from warehouse to mailbox) are the most expensive part of delivery for businesses like Amazon. They also takes the greatest toll on society – using fossil fuels to power trucks, creating traffic in already congested areas, and filling landfills with the extra packaging needed to protect the goods during transit. Drones like the one we created for our senior project are able to solve all of these problems, helping both the businesses and society as a whole.

What It Does

This multirotor drone uses computer vision, GPS, and other sensor data to autonomously transport packages from a warehouse to a delivery destination, safely drop the package off, and then return to the warehouse for the next delivery.

How We Did It

My team designed this drone from the ground up before ultimately building and testing it. The first step was to concretely lay out our requirements and design specifications. Our original goal was just to make a working delivery drone, but after researching the cost of parts, we decided to also strive to make the whole unit cheaper, as this would be a product with large volumes.

Researching for this project was by far the biggest and most in depth analysis I have ever done. Together, my team contributed over 1000 hours of research to this project. Because the project was so open ended, and there were so many good options to use in our drone, it took a lot of work to select the perfect components for our system.

At this point, drones, are so widespread and readily available, that our airframe was not the interesting part of the project. We did not do any special designing for this. We simply picked an airframe that would fit our needs, namely be able to carry a 0.5 pound package 10 miles round trip. We picked this number because a huge number of packages (over 2/3) are able under 0.5 lbs.

At this point, drones, are so widespread and readily available, that our airframe was not the interesting part of the project. We did not do any special designing for this. We simply picked an airframe that would fit our needs, namely be able to carry a 0.5 pound package 10 miles round trip. We picked this number because a huge number of packages (over 2/3) are able under 0.5 lbs.

The most interesting part of the project, and my specialty, was the computer vision. This is where I spent the majority of my time researching, testing and implementing. I decided that computer vision was a good idea for this project because lidar is so much more expensive. After much investigation, I concluded that SLAM (Simultaneous Localization and Mapping) would be the best computer vision algorithm for our purposes. This would allow us keep track of the drones position and orientation in space over large distances, as well as remember where it had been. It would also map obstacles in the environment to ensure that the drone does not collide or crash. For the project, we used ORB-SLAM 3.0. This is an open-source SLAM implementation that had the perfect balance of performance, features, and robustness. We ran this on a Raspberry Pi on top of a special real-time versions of Linux.

Our flight control software was PX4, another open-source platform. It was relatively simple to set up and run on the Raspberry Pi. The challenging part was connecting it with our computer vision layer. When the camera detects an obstacle, how do we navigate around it? PX4 comes with a package that enables obstacle avoidance using a point-cloud. The problem is that it uses it’s own computer vision software and requires a stereo camera with depth sensors. These are very expensive, and we avoided them for the same reason as the lidar sensors. This meant that we had to modify the PX4 Obstacle Avoidance package to skip its own computer vision algorithm and instead take in the point-cloud from ORB-SLAM. This was very challenging and finnicky to get these programs to work together. We ultimately used ROS to communicate between them. This allowed us to finally do obstacle avoidance with a single camera.

What I Achieved

The biggest accomplishment of the project was reducing the system cost by 40% through the use of computer vision rather than lidar or other expensive sensors. At the time of the project, the appropriate lidar sensors costed around $300. With the whole system costing around $700, this is a huge expense. Using a $10-$20 camera, we were able to reduce the unit cost by a huge amount, which helps to manufacture this product at a large scale and deploy it widely.

Another part of the project I’m proud of is being able to connect many different software into a complete system. Many parts of the project were trivial to set up on their own. The challenge came from integrating them together. I am proud of the fact that I was able to get each part working with all the others, especially the obstacle avoidance software that I had to modify.

What I Learned

This project was a very open ended research project that required lots of planning and problem solving to complete successfully. With this being very similar to my Real Time Systems Security Research, I strengthened and improved many of the skills I learned from that project, namely: developing concreate requirements, researching complex open ended topics to get results, and designing a complete software system.

In addition to these skills, I also learned a lot from writing the research paper that went along with this senior project. It taught me how to clearly document technical details and effectively communicate ideas and impact in a way that non-technical stakeholders can understand.

Lastly, I strengthened my ability to integrate separate programs into the whole. This is something which is very important in creating a finished product. While I did gain some practice doing this in my other research projects, the delivery drone gave me much more of this valuable experience.

Overall, this project has made me so much more confident with my general engineering ability, in addition to my programming skills.

• What Is It? – Testbench used to train AI to detect security threats in safety-critical real-time systems
• Technologies Used – Java, FreeRTOS, MicroROS, Spike (RISCV-ISA-SIM), FPGA
• My Accomplishments
  • Developed concrete requirements by understanding motivation and goals
  • Designed and implemented RTOS resource monitor testbench in software
  • Successfully planned and transferred testbench from software to hardware, including physical coprocessor

Motivation

Historically, security has not been a priority in real-time systems, but this is not for lack of need. Real-time systems are becoming more common in safety-critical applications such as self driving cars. In these systems, malicious attacks can have serious, sometimes deadly, consequences. Implementing a security solution for real-time systems is not trivial, however. Embedded systems do not have extra compute power to devote to complex security programs like desktop computers do. A security solution that does not impact performance is desperately needed for these safety critical systems.

Background

• RTOS: A real-time operating system (RTOS) is an operating system for real-time computing applications that processes data and events which have critically defined time constraints.
• Mixed-criticality system: A system in which some tasks are high-criticality (must be completed) and some are low-criticality (acceptable to not finish)
• Mode-switch: When a low-criticality task overruns its deadline, the OS stops execution of low-criticality tasks and only executes high-criticality tasks. This is to ensure that the high-criticality tasks will finish by their deadline.

This research project was based on the paper “Pythia-MCS: Enabling Quarter-Clairvoyance in I/O-Driven Mixed-Criticality Systems.” This paper uses a coprocessor to measure resource usage statistics and I/O volume. With this information, an AI is trained to predict task overruns and mode-switch before they occur. Doing this increases schedualability in the system.

What It Does

Our project extends this idea of overrun prediction by analyzing many different metrics such as resource usage and memory access patterns in real time. Using a coprocessor, our system detects and terminates security threats before they are able to cause damage.

How I Did It

My role in this project was to architect and implement the technology stack we would use. After laying the foundation, my professor and his graduate student would use my setup to test and develop the security coprocessor. I did my part in three phases.

First, I gathered requirements. This was relatively easy to do, because my team already had a good idea of the goals for the project. After a few discussions, we settled on our final requirements. I then used these to design an optimal system. My plan was to first create a proof-of-concept completely in software, as it is easier to iterate with. Once that was promising, we would then transfer it to the physical board.

Second, I created the software testbench. I used Spike, which is an open source, RISCV instruction set architecture simulator. It would simulate the physical instructions that our processor would run. On top of this simulator, I ran FreeRTOS. This is an open source real-time operating system. This is a very common OS used in embedded systems. Lastly, on top of FreeRTOS, I ran microROS, which is a a commonly used open source robotics library. This would allow us to run our test cases easier. We could run microROS operations and then analyze the resource usage. To simulate the coprocessor, I wrote a python program. The program would essentially be a driver for Spike. It would have Spike execute one instruction, then analyze the output. I had the program log things like register values, memory addresses, instruction types, and I/O volume. From here, my team could use my tool to gather and analyze any information they wanted.

Last, I set up the physical board. The hardware we decided on was the Xilinx Zynq-7000. This was perfect for our needs because it has an ARM CPU and FPGA in the same chip. Because they are on the same chip, the FPGA has access to the CPU’s hardware such as registers. The team would use the FPGA to create a coprocessor in hardware, while still being able to quickly iterate. All that was left was to install the software stack onto the board and then design the coprocessor. This is where I handed off my contribution to the others, and allowed them to get started on the analysis.

What I Learned

I learned to develop concreate requirements. Unlike school projects, the requirements were not handed to me. I had to understand the motivation and goals of the project, and come up with requirements that allowed us to achieve the goals. One thing that was important for me was to avoid thinking of implementation. This puts too many constraints on the project and can impede the goals. I learned that it is important to focus on the purpose of the project and leave implementation for later.

I learned how to research a complex, open ended topic to get results. There were many different directions I could have gone with this project, and I ran into many dead ends. It was discouraging but a valuable learning experience. I learned to do preliminary research on a wide area and slowly narrow the scope as I learned more about my options.

Lastly, I learned how to design a complete software system. Figuring out the overall architecture of a project, and how each piece will fit together, is much more challenging that writing a self contained program. I learned to consider how each piece fits together, which is something that came in handy with my Autonomous Delivery Drone project.

Overall, this project taught me a lot about the design and planning that goes into a large complex project.

• What Is It? – Program to analyze cache behavior based on memory access patterns
• Technologies Used – Java
• My Accomplishments
  • Gained understanding of physical memory and cache operation
  • Studied how access patterns impact hit-rate and access time
  • Applied Object Oriented Programming best practices

What It Does

The program simulates a series of memory accesses while keeping track of key performance statistics like hit rate and hard disk accesses. When a memory access is requested, the program first checks the page table to see if that address is in TLB. If there is a TLB miss, the program moves on to check the page table. If that address is still not found, that means that it must be on the hard disk. The program simulates reading the block from the hard disk and finds an open frame in the frame table to put the block. The simulation puts the entry into the page table and marks the corresponding frame as allocated.

Finally the translation is moved from the page table to the TLB to simulate caching the data. If any of these tables are full, an eviction is done before moving the data. The simulation implements a fully associative mapping, the write-back scheme, and the least recently used eviction method.

How I Used It

I used this program to study how page size and cache size can impact the performance of a computer. In my case study, I tested 1000 random memory accesses of 8 pages versus 1000 random memory accesses from 9 pages. This seems like a tiny difference, but the outcome changed dramatically. The TLB hitrate dropped from 99% to only 87% because the data could no longer completely fit in cache.

Additionally, I looked at how code localization can affect the speed at which that code runs. I simulated code jumping outside of its current block 25 times versus 400 times. This resulted in a hitrate decrease of 7.5% which would cause the code to execute more slowly.

What I Learned

Before writing this program I thought I had a good understanding of how virtual memory worked. In development I realized all the parts I overlooked. By the time I finished, I had gone over every detail and intricacy of the virtual memory hierarchy that I would consider myself an expert. Additionally, I learned the importance of understanding a project’s requirements. Many school assignments can be learned during development, but more complex projects need to be planned out and fully understood before starting. I also gain a better grasp of object-oriented best practices like abstraction and minimizing side effects.

Link to Code

Because this is a school project, I had to make the GitHub repo private. This link gives read access to the repo without making it public. Virtual Memory Simulation Code