Gigapixel imaging: When bigger IS better

2022-11-15 Steve Feller

When developing a new, and potentially transformative technology, it is easy to be overly optimistic both about it's impact and seemingly obvious benefits for existing customers and applications. This was certainly the case when I started working on the high resolution, wide field of view Mantis systems at Aqueti. And while I learned a lot from that experience, I find myself in the same mindset as I think about the gigapixel microscopy systems being developed at Ramona Optics. While vastly different solutions, they both generate gigapixel-scale datasets that provide unprecedented levels of detail across a very wide field of view. The value proposition of these systems is intrinsically tied to the images they generate, or more specifically, the wide field, high resolution information they can acquire.

While gigapixel-scale images contain orders of magnitude more information than a data acquired from conventional cameras, they are not the perfect solution in all cases. These systems are complex, expensive and generate significant amounts of data that must be processed, stored, or viewed to be useful. The goal of this post is to distill down the known applications where these systems have been successful to develop a general framework for understanding when and where imaging at this scale makes sense and in what contexts these images bring value to a given application.

Gigapixel Imaging Systems and their applications

Known use cases for gigapixel imaging systems that provides important context for this discussion. The systems highlighted below target very different applications. The Aqueti system was specifically developed for wide-field, high resolution security and event capture while the Ramona Optics builds gigapixel-scale microscopy systems. While I have a passing knowledge of other high pixel count solutions, these are the two that I am most familiar with and have thought a lot about.

The Gigapixel Value Proposition

While the Aqueti and Ramona Optics systems have been developed for very different applications, there are fundamental aspects of gigapixel imaging that are relevant in both cases. In both cases, it is important to understand the value imaging at this scale and the specific uses cases where it provides significant advantages over existing systems. While there are many ways to break down this analysis, I would argue that these uses cases can be generalized into the categories outlined below.

Detecting rare events

There are many cases where it is important to capture a rare or unexpected event. This use case is obvious for security cameras, which are specifically designed to detect when people are where they are not supposed to be or are doing something that may be illegal. This also the use case for imaging freely swimming Zebrafish in a microscope. In these applications, the combination of a wide field of view and high resolution makes it possible to detect these events whenever and and wherever they occur.

Detecting distributed, simultaneous events

Measuring simultaneous events in high resolution across a wide area is difficult for conventional cameras with limited resolution and field of view. Gigapixel imaging systems are uniquely suited for these applications given the pixel density across the entire field of view and the ability to take a snapshot in time. In security applications, this is important when two simultaneous intruders enter a security setting. For microscopy this can be used to measure the impact of external stimuli on multiple sample organisms or to analyze sample evolution in multiple well plates simultaneously.

Post processing and forensic analysis

The sheer amount of the data acquired by gigapixel-scale imaging systems in a single frame is beyond the capabilities of a conventional camera. Capturing a snapshot of the entire scene at this scale and rate provides new opportunities for post processing and data analysis. Potential applications of this post processing and analysis include working backward from the time a zebrafish heart stops beating to establish the conditions leading up to the failure. For security, this could include working backward from when an something significant happens to see the events leading up to it. In both cases, it is not clear what is important at the time the data was acquired, but accessing the data after the fact can provide valuable information.

Increased Throughput

Camera array systems are often compared directly with scanned single camera solution. If the sample or area being imaged is static or only changes at rates below the scanning frequency, then the primary difference in the systems is in how quickly the entire sample can be image. For example, early gigapixel-scale panoramas were acquired with a single camera mounted on a mechanical pan/tilt stage and programmed to acquire images across the entire space over minutes, hours or even days. An Aqueti system could take the same image at multiple frames per second by taking the images in parallel. A similar example in microscopy would be using a conventional microscope to sequentially image multiple wells in a large well plate. The primary drawback of this scanning approach is that the acquisition time may be orders of magnitude longer than a single image from a Ramona Optics system. This limits both the number of samples that can be acquired in a given period of time and makes consistent image quality difficult in dynamic environments. By imaging in parallel, gigapixel scale systems provide faster, more consistent images.

Conclusion

Identify the the contexts where gigapixel scale imaging systems have a clear impact is an important step in understanding their value to the customer. This analysis identifies four main areas where imaging at this scale provides significant advantages over conventional systems. These areas include looking for rare events, viewing simultaneous events that are not co-located, forensic analysis, and increased throughput over lower resolution solutions. Discussing the capabilities and limitations of gigapixel-scale imaging systems in a general sense allows experts in diverse field to understand their potential in domain specific applications. Now that we have established some context for where these systems are useful, it is time to think broadly about the applications that would benefit from these capabilities. I look forward to a detailed discussion about potential uses for these systems either in the comments of this post or through direct communication.