• Color Grading & Restoration
  • Blog
  • ABOUT
  • Contact
Menu

Zachary Halberd

  • Color Grading & Restoration
  • Blog
  • ABOUT
  • Contact

Computer Vision + Drone Imagery = Smarter Cities

July 22, 2022

In 2017 I co-founded a startup called Quantifly and worked for several years as a manager and technical consultant. We were accepted into the Genius NY program which is the largest accelerator in the world for drone technology. We took second place at the finals and received $600,000 in funding along with several other grants and support. I helped Quantify design, build, and implement a complex and sophisticated aerial data collection and analysis pipeline. I hired a team of engineers and researchers to develop a computer vision-based algorithm for identifying cars from aerial imagery. The algorithm was also able to detect change between subsequent temporal layers. I was in on the ground floor and followed the entire process through to successful completion. I flew multi rotor and fixed wing drone systems, built and tested custom camera payloads, installed and tested extremely precise real-time geolocation and tagging computers, and created the image processing pipeline which fed into the CV algorithm. It was the most technical and challenging opportunity of my life and I’m always thankful for the experience.

naj-7638.jpg
slide27.jpg
DJI_0531.JPG
20180912_200701.jpg
DJI_0806.JPG
naj-7638.jpg slide27.jpg DJI_0531.JPG 20180912_200701.jpg DJI_0806.JPG

When I began to experiment with drones back in 2011, I quickly realized their market potential expanded far beyond the film industry. Drones (or UAS as the FAA calls them) are machines which can move in three dimensional space, have the ability to carry sophisticated payloads, and can be automated. To a drone pilot, the world is full of potential. (I have more to say about the regulatory environment and its relationship to free enterprise within public airspace but you’ll have to buy me a beer to hear that)

I participated as a co-principle investigator on several research projects at Wayne State University including one paper titled “Channel Fading Statistics For Real-Time Data Transmission In Emergency Call Systems And Unmanned Aerial Systems” with Dr. Ynrui Li for his doctoral dissertation. I modified one of my commercial off the shelf drones to provide a robust testing platform in sub-zero temperatures in an urban environment for cellular signal path testing. Our submission to IEEE was rejected but the dissertation was a win! I also collaborated with another team of engineers and doctoral candidates at Wayne State to build and test an aerial platform for water quality testing. The concept included flying the drone in an automated pattern and dipping an instrument (IoT) into the water before transmitting the data back to the server in the form of an SMS using Arduino. We submitted our idea at a TechTown Hack Lake Erie hosted by the City of Detroit and NASA but ultimately did not win the competition.

 

Unmanned Aerial Systems Projects

Aerial Photogrammetry for Parking/Traffic Analysis

I designed, built and tested several unmanned aerial imaging systems for Quantifly LLC to provide the novel dataset necessary for parking and traffic analysis. I used these systems to map the downtown areas of several major American cities. One of our missions was to map the entirety of the new Detroit Riverwalk North of the GM headquarters. We then use a proprietary process to prepare the images for high precision photogrammetry for automated analysis.

OPERATION_PIONEER_02.jpg
AS_AERIAL_FRAME_GRABS_00001.jpg
DJI_0806.JPG
DJI_0531.JPG
GALAPAGOS_SCAN_WEB.jpg
ROYAL_OAK_PS_19_JULY_2017_110M_-65.JPG
OPERATION_PIONEER_02.jpg AS_AERIAL_FRAME_GRABS_00001.jpg DJI_0806.JPG DJI_0531.JPG GALAPAGOS_SCAN_WEB.jpg ROYAL_OAK_PS_19_JULY_2017_110M_-65.JPG

Aerial Real-Time Kinematic Geolocation Image Tagging

Quantifly wanted to be able to overlay their orthorectified 2D aerial maps of urban areas on top of existing GIS data provided by municipalities in order to use as an additional layer for reference. This meant that I had to figure out a way to embed calibrated geolocation tags into the metadata of every image taken by the drone during the mapping process. I took a course in basic surveying and went to work with an RTK

20180608_205016.jpg
20180608_161025.jpg
20180608_182859.jpg
20180611_124638.jpg
20180707_194245.jpg
20180608_205016.jpg 20180608_161025.jpg 20180608_182859.jpg 20180611_124638.jpg 20180707_194245.jpg

Aerial Telephoto System for Tower Inspection

I modified a Micro 4/3 gimbaled sensor system to accommodate a 300mm zoom lens for aerial inspection of RF equipment and structural integrity inspections. I also developed the protocol for remote drone operators working in different regions for quality control.

TOWER_INSPECTION_TEST_01_0025.jpg
TOWER_INSPECTION_TEST_01_0056.jpg
Screen Shot 2022-07-21 at 6.03.52 PM.png
ALPHA_SITE_2623_179.JPG

UAV Water Quality Testing With Sweetly

After several years of experimenting with drones on my own, I decided to get my commercial UAS pilot license from the FAA. I began flying drones for television series and movies in NYC and Detroit. It’s good money but can be super stressful at times. Its a lot of travel and requires many skillsets. I loved it. Then fate fell into my lap. I was in Detroit attending a Hackathon and met a data scientist that specializes in water quality. Dr. Javad Roostaei had formed a small team of researchers at Wayne State University to connect IoT devices which monitored water quality in Lake Erie to a database which the public could access. I pitched the idea of using a drone and the rest is history.

_DSC0108.jpg
_DSC0088.jpg
20180611_210114.jpg
DJI_0007.jpg
DJI_0015.jpg
slide12.jpg
slide4.jpg
_DSC0108.jpg _DSC0088.jpg 20180611_210114.jpg DJI_0007.jpg DJI_0015.jpg slide12.jpg slide4.jpg

AUVSI and SwissNext Conferences

I was fortunate enough to speak at both of these prestigious Unmanned Aerial Systems and Robotics conferences on behalf of Quantifly. I was also given the opportunity to host a career and education conference with several engineering firms in upstate New York. The program paired hosts from the tech industry with students from Syracuse University and Rochester Institute of Technology in order to facilitate discussion about privacy and security in a future with aerial robots and urban data collection.

43718799660_f0e595ebb1_k.jpg
30595476657_a037f47e2b_k.jpg
45485673182_14cda9c641_k.jpg
43718799920-dec53166bf-k_orig.jpg
20180413_105146.jpg
IMG_0204.jpg
43718799660_f0e595ebb1_k.jpg 30595476657_a037f47e2b_k.jpg 45485673182_14cda9c641_k.jpg 43718799920-dec53166bf-k_orig.jpg 20180413_105146.jpg IMG_0204.jpg
← DIY Slide Scanner for End to End Color ManagementThe Rock - The Physiological Simulation of Dining in Zanzibar →

Latest Posts

Featured
DOC_WORKFLOW_20240428.png
May 12, 2024
Documentary Finishing Workflow Demystified
May 12, 2024
Read more →
May 12, 2024
Using AI Models to Colorize Monochromatic Archival
September 30, 2022
Using AI Models to Colorize Monochromatic Archival
September 30, 2022
Read more →
September 30, 2022
Archival Restoration - Experimenting with Python Using VapourSynth + Resolve
October 25, 2021
Archival Restoration - Experimenting with Python Using VapourSynth + Resolve
October 25, 2021

Take a peak under the hood of my process to restore a heavily compressed archival film using the open source Python script-based image processing tool Vapoursynth.

Read more →
October 25, 2021
Qualitative Camera Test v2 - Summer 2021
October 25, 2021
Qualitative Camera Test v2 - Summer 2021
October 25, 2021

I convinced ten cinematographers to bring their cinema cameras to a qualitative camera test. Here are the results.

Read more →
October 25, 2021
axel-vazquez-nph6DcLTckI-unsplash.jpg
July 3, 2021
Premiere to Resolve and Back Again - A Remote Workflow Guide
July 3, 2021

For the love of God, please stop using shuttle drives. Just use the Resolve media manager to shrink ray your Premiere timeline down to a tiny portable file and send to your colorist!!! In this article, I show you how!

Read more →
July 3, 2021
VapourSynth Installation for Python 3.6.8
June 14, 2021
VapourSynth Installation for Python 3.6.8
June 14, 2021

My ongoing experiments with incorporating the Python-based image processing software, VapourSynth, into my color management and restoration pipeline has resulted in compiling a stable package which Windows users may use for further experimentation.

Read more →
June 14, 2021
Deconvolutional Upscaling for Legacy Media
February 9, 2021
Deconvolutional Upscaling for Legacy Media
February 9, 2021

MCM Creative in NYC reached out to Magic Lantern and asked if it was possible to clean up and upscale his feature doc Shelter Island prior to an Amazon streaming distribution sale. Unfortunately, the original media was no longer available and the master had already gone through various unknown encodes.

Read more →
February 9, 2021
Superior Deinterlacing and Upscaling With Machine Learning
February 2, 2021
Superior Deinterlacing and Upscaling With Machine Learning
February 2, 2021

Typically, filmmakers use the deinterlacing tool provided for free within their NLE or Davinci Resolve. This article will demonstrate why this approach can be destructive to the image and will offer a better alternative.

Read more →
February 2, 2021
Rendering With JPEG Compression(ism)
January 28, 2021
Rendering With JPEG Compression(ism)
January 28, 2021

A collection of photos taken with a cell phone at full magnification yields interesting results. Although the discrete cosine transform compression still has a significant effect in the image, by magnifying the image in camera prior to the encode, these images take on a hybrid of pixilation and compression decimation. There is also a slight halation ghosting around the higher contrast edges which get encoded into a pleasant glow.

Read more →
January 28, 2021
Color and the Physiological Response - A Colorist's Exploration
January 28, 2021
Color and the Physiological Response - A Colorist's Exploration
January 28, 2021

Colors, generally speaking, are instinctively associated with emotions. Working as a colorist and having to manipulate color everyday – I feel that most directors and cinematographers intuitively ask for specific colors to enhance specific emotions on special scenes without even realizing it. Here is a little help on how to actually use color in film making to activate – or not – a physical reaction in your audience.

Read more →
January 28, 2021