Sensor fusion discussion on LinkedIn by Spencer:
Let me present some "reason to believe."
First, probabilities "add." In other words, if sensor A readings have a 40% level of confidence, sensor B readings have a 25% level of confidence, and sensor C readings have a level of confidence of 45%; one now has a 100% level of confidence that "something" is there that should, or should not be there.
Now the economics get interesting. To get 98-99% level of confidence using ONLY sensor system A, B, or C costs, eg. $5,000, what does an AI sensor fused A, B, and C cost? In our admittedly limited experience, say only $500-1000 for a 100% confident determination of an intruder, etc.
We learned this over our last 15 years of R&D. The easiest example to understand is our GeckoOrient(tm) solution. http://www.geckosystems.com/low_level/geckoorient.php This AI leveraged solution gives us +/- 1/2 degree orientation accuracy with no accumulated error in a very lost cost and portable system.
There is a part 2 to this discussion. These multiple, enhanced sensor systems should be on a mobile service robot, as a peripheral on the IP "video" LAN/WAN.
As is generally known, fixed visible light video systems, even with PTZ, can be thwarted. Mobile with sound detect, passive IR detect, ....
Why will this type of augmented IP video surveillance be more cost effective that using people and dumb video to surveil 100% of the time?
Humans have limited vision capabilities. We do not see infrared, or any of the electromagnetic spectrum. In order for a human to see IR, the IR must be transformed into visible light images for a human to discern. That need for converting IR images to visible light images costs money. When AI software is used to interpret an IR image, there is no need to to incur the cost of that invisible light to visible light conversion.
Object tracking for machine vision can be simplified using Passive IR detect (for warm human bodies) and distances determined using ultrasonic range finding. Using this AI sensor fusion approach, a warm body is detected by passive IR, the distance to the Mobile Service Robot (MSR) is determined by sonar, and the onboard speech synthesis can ask "who are you" since it is 3:30A and no one should be on this parking lot for BMW's, etc.
Now the watch commander that oversees a squad of sensor augmented MSR's, is paged by the MSR spotting someone at 3:30A at a parking lot that should be deserted.
The watch commander would get very few "false positives" in this scenario. The watch commander can immediately take control of the PTZ on board and converse with the "intruder" as to what are they doing there at 3:30A?
The horrific massacre at Sandy Hook is what got us to thinking about applying our AI and sensor fusion tech in this way.
So far I have discussed how poorly the unaided human eye sees its environment and that it costs to convert invisible light images to visible for human viewing.
There is a similar set of circumstances in "hearing." First, as a general rule, men do not hear as well as women in the higher, audible frequencies. Further, amplification of distant sounds can easily create sound pressure levels painful to normal human hearing. This is not a problem in AI interpreted "listening" systems.
Also direction of the sounds is difficult for humans to sometimes discern. In an AI interpreted 'listening" systems, all sound frequencies, including high frequencies, above the human range, can be analyzed without conversion to lower frequencies audible to the human ear. Here I believe that shotgun microphones and/or parabolic microphones, and directional microphone arrays can be AI fused to produce pretty good levels of confidence that "something" made a noise in "that" direction.
Here's an example of the intruder detect payoff-
Suppose passive IR indicates a warm body at 30 degrees to the left with a 45% level of confidence. Machine vision object recognition indicates that at 32 degrees to the left with a 30% level of confidence there is a "blob." Sonar indicates nothing at 28-32 degrees, but range limited to 50 feet. Sound interpretation indicates "noises" from 25-35 degrees to the left with a 35% level of confidence (aka probability). Since probabilities add, we are now 100% sure there is something warm and making noise somewhere between 25-35 degrees to the left and probably further than 50 feet.
Bingo! The watch commander is notified by the AI that "something" seems to be there-
The watch commander can now take over the MSR with a joystick and, at potentially 15-20 mph, quickly "run' in the direction initially indicated. As the MSR gets closer, the passive IR indicates a higher level of confidence and the AI listening now hears a person breathing due to the amplification and directionality available. The intruder can now be asked by the watch commander through the on board PA and video conferencing system, "Who are you? And what are you doing here at 3:30A?
The cost effectiveness of this scenario should be very attractive since we are discussing 7/24/365 coverage by machines that take no breaks, no vacations, no time off, no training, no scheduling issues, low maintenance, and no annual 200-300% turnover in security personnel. In other words, where one once had 21 bored, short term security guard employees. One now has one watch commander per 8 hour shift (at say 150% of guard pay), or the cost equivalent of 4.5 of the old 21 guards. This leaves 16.5 guards' fully burdened pay to offset the cost of this AI sensor fused mobile security robot. One would think that $320,000 or more in annual savings will purchase and maintain 4-6 MSR's here.
Hence this MSR scenario for security will probably pay for itself in one year or less...
These are MOBILE video cameras on automatic, no routine human intervention patrol with much more situational awareness than simple visible light machine vision interpretation.
The camera costs are reduced due to mobility and sensor fusion.
Sorry if I have been unintentionally obtuse.
In a way, you make my point that when a singular sensor system is pushed to provide actionable situation awareness, the costs explode...
If I may, we do have some heritage in AI sensor fusion cost/benefit analyses-
