1) SIFT vs ORB vs SURF Feature Detectors: Paper
- Both SIFT and ORB are available on opencv and are open-source
- We found SIFT was better able to keep points after image transformations than ORB
- However, SIFT was unable to attain realtime performance, so the ORB feature detector was used
- PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image Paper
- Relavent and cool paper about plane detection from a single RGB image w/o depth data using neural networks
- Capabilities of ARCore and ARKit Platforms for AR/VR Applications: Pa 74F9 per
- Performance stats of Apple and Google's implementations for AR programs
- CLAHE: Apply Contrast equalization to image's to extract more features. This is useful for getting features from flat & smooth surfaces.
- Use ORB feature/descriptor detector to detect and compute feature points from CLAHE'd image
- Compare descriptors from current and following image using matcher + lowe's ratio test to match keypoints
- Apply RANSAC to throw out outliers and clean up motion flow
- Find the Homogrophy; which gets the transformation/orientation data of a surface
- Draw 3D object on detected surface
- Plane Detection without CLAHE Equalization
- Plane Detection with CLAHE Equalization:
- The 2nd demo is able to detect flat surfaces but is slow (i.e. non-realtime) and could be improved upon
- Integrate fully with web browser via Javascript
- Improve the js pipeline to increase fps streaming. Capped at 1-12 fps currently
- Implement CLAHE in some regions only to improve speed and fix glare issues