Make color camera parameters adjustable

[charisoudis]

Hi, and thanks for the great work.

I am using an iPad along other sensors to capture a dataset under studio conditions. I.e. the capture volume is mostly dark with wide diffuse lights glowing towards the area of interest.

iPad color camera seems to significantly overexpose the image, leading to color images that are almost unusable even after reducing the exposure post-capturing (e.g. see image below after reducing the exposure, vs an image from an RGBD camera from ORBBEC with reduced exposure settings).

Is there any control over the color sensor settings (e.g. focus, exposure, white balance)? If yes, are there options already in the UI / iOS settings?

Best regads,
Thanos

iPad:

ORBBEC:

[kekeblom]

At the moment, the UI doesn't expose any exposure, focus or other image settings, but this would indeed be a nice feature to have. Especially toggling between fixed focus, setting the focus as well as fixed exposure, setting the exposure params would be super nice.

This should be possible to do via the API. The trickiest part would be to make the fit smoothly into the UI. PRs are welcome on this one!

[jerredchen]

Hi,

For my own project, I'm also interested in accessing the exposure time of the sequence (though I personally don't need to adjust it). I have a very small change that I can make in the OdometryEncoder if that would be of interest? If so, I can create a small PR for this.

//  OdometryEncoder.swift
//  StrayScanner
//
//  Created by Kenneth Blomqvist on 1/31/21.
//  Copyright © 2021 Stray Robots. All rights reserved.
//

import Foundation
import Accelerate
import ARKit

class OdometryEncoder {
    let path: URL
    let q_AC = simd_quatf(ix: 1.0, iy: 0.0, iz: 0.0, r: 0.0)
    var transforms: [simd_float4x4] = []
    let fileHandle: FileHandle
    
    init(url: URL) {
        self.path = url
        do {
            try "".write(to: self.path, atomically: true, encoding: .utf8)
            self.fileHandle = try FileHandle(forWritingTo: self.path)
            // >>>>>>>>>>>>>>> Here I added a new column header for the exposure
            self.fileHandle.write("timestamp, frame, x, y, z, qx, qy, qz, qw, exposure\n".data(using: .utf8)!)
        } catch let error {
            print("Can't create file \(self.path.absoluteString). \(error.localizedDescription)")
            preconditionFailure("Can't open odometry file for writing.")
        }
        
    }

    func add(frame: ARFrame, currentFrame: Int) {
        let transform = frame.camera.transform
        transforms.append(transform)
        let xyz: vector_float3 = getTranslation(T: transform)
        let q_WA = simd_quatf(transform)
        let q: vector_float4 = (q_WA * q_AC).vector
        let frameNumber = String(format: "%06d", currentFrame)
        // >>>>>>>>>>>>>>>> I access the exposure through the API and record it in the file
        let exposureDuration: Double = frame.camera.exposureDuration
        let line = "\(frame.timestamp), \(frameNumber), \(xyz.x), \(xyz.y), \(xyz.z), \(q.x), \(q.y), \(q.z), \(q.w), \(exposureDuration)\n"
        self.fileHandle.write(line.data(using: .utf8)!)
    }

    func done() {
        do {
            try self.fileHandle.close()
        } catch let error {
            print("Can't close odometry file \(self.path.absoluteString). \(error.localizedDescription)")
        }
    }

    private func getTranslation(T: simd_float4x4) -> vector_float3 {
        let t = T[3]
        return vector_float3(t.x, t.y, t.z)
    }
}