ARKit materials have a .blendMode property that allows you to overlay the color of your rendered object on top of the video image. This works great, however, I need to use a different blend mode to apply the color to the video image. I have tried creating a custom SCNTechnique that applies my custom blend mode, and the shader works, but in order to blend the render objects color with the original ARKit video feed, I need access to the image before rendering. I have this working with the mask created from people occlusion in Metal rendering, but not with my own geometry. Additionally, I'd prefer to keep things in ARKit with the normal SceneKit rendering. Does anyone know how I can access the original ARKit video images in a custom SCNTechnique shader within the bounds of SceneKit? How I imagine this working is as part of the SCNTechnique Passes, I render the original video to a texture or pass it in somehow, then render the geometry to a different texture, then on a third pass, use both of those textures to compute my final image.
How to access the original ARKit camera image in a custom shader or SCNTechnique?
186 Views Asked by Davido At
0
There are 0 best solutions below
Related Questions in SWIFT
- Overlapping UICollectionView in storyboard
- Cannot pod spec lint because of undeclared type errors
- Swift code with multiple NSDateFormatter - optimization
- How do I add multiple in app purchases in Swift Spritekit?
- cellForRowAtIndexPath and prepareForSegue return different label colors
- Getting this message in my console in xcode "Ignoring restoreCompletedTransactionsWithApplicationUsername: because already restoring transactions"?
- Change background of an Accessory View in a UITableViewCell
- fade in an bounce animation subview
- Create a PFObject and PFRelation after PFUser Sign Up
- Swift 2 - Pattern matching in "if"
- How do I give inputs through NSURL
- How do I add custom cells to TableView in Swift?
- UIWebView not loading URL in simulator
- Compiler complains that 'Expression resolved to unused function' when removing index in array of functions
- Cast from 'Int?' to unrelated type 'NSNumber' always fails
Related Questions in SCENEKIT
- Is there an equivalent for SCNAction, SCNTransaction or SKAction in Unity?
- How to setup camera to point at object
- How to convert from ModelIO MDLMesh to SceneKit SCNGeometry and back?
- Scene Kit: projectPoint calculated is displaced
- Adding audio player to a node in SceneKit
- How do I use multiple textures in an OpenGL SL shader in SceneKit in iOS?
- SceneKit + Collada + animation
- Is there a way to use height map with SceneKit?
- Cloning nodes and performance, geometry, and performance; IOS Scenekit
- SceneKit shader modifiers with GLSL arrays
- Cylinder Orientation between two points on a sphere, Scenekit, Quaternions IOS
- Using a Metal shader in SceneKit
- Spin box/camera around with velocity in SceneKit
- Scenekit PhysicEngine follow rolling ball
- How to follow SCNNode with SCNCamera but don't rotate?
Related Questions in ARKIT
- How to create a DAE file to use in SceneKit?
- How to store additional information in SCNNode?
- How to find obstacles in horizontal surface using ARKIT of iOS 11(Scenekit)?
- SceneKit - Rotate object around X and Z axis
- Using SCNFloor in ARKIt
- Draw straight line from start value to end value ARKit
- ARKit drop a custom object in the scene
- How to use DAE file with different Scene Graphs in iOS SceneKit
- How to place a 3D object above the other one using ARKit?
- How to detect collision between nodes of two different .scn files in Scenekit- iOS?
- Reliable access and modify captured camera frames under SceneKit
- Swift: How to set size of particle effects with SCNParticleSystem?
- Scanning a 3d object in ARKit via video camera?
- ARKit and Vision frameworks for Object Recognition
- How do ARCore or ARKit produce real-time augmentations of live video?
Related Questions in METAL
- How to get fragment coordinate in fragment shader in Metal?
- Porting OpenGL app to Metal API
- Using a Metal shader in SceneKit
- mathematical movable mesh in swift with SceneKit
- Z-Value, Z-Near and Z-Far Within Kernel Function
- Filling Float buffer in Metal
- Metal on iOS: `newCommandQueueWithMaxCommandBufferCount` not working
- Using the [[clip_distance]] attribute in a Metal shader?
- iOS: How to manually set a 2D Float data to MTLTexture or MPSImage?
- Loop unrolling in Metal kernels
- What is Explicit synchronization between GPU and CPU
- Why can't I bind an asset once in metal?
- Apple Metal without Interface Builder
- Metal Framework on macOS
- Implement a custom layer after a series of MPSCNNConvolution
Related Questions in SCNTECHNIQUE
- How to use stencils with SCNTechnique?
- Metal / SceneKit Fragment Shaders - How to avoid rendering ontop of other geometry?
- How to add "excludeCategoryMask" for two different Categories in the same pass?
- iOS SceneKit SCNTechnique Metal vertex and fragment methods not work in iOS12
- How add custom SCNTechnique pass to specific area, without messing with the rest of Sceneview fullscreen area?
- How to use SCNTechnique to create a "masked" portal effect in SceneKit / ARKit?
- Blank depth buffer from ARSCNView in DRAW_QUAD pass of SCNTechnique
- How to access the original ARKit camera image in a custom shader or SCNTechnique?
- Modify Metal fragment shading based on vertex world position Y coordinate
- Multiple SCNView audio error
- SCNProgram read from depth buffer
- How come SCNTechnique doesn't have a .library property like SCNProgram?
- Motion Blur with SCNTechnique
- Low cost Image to NSData conversion for feeding SCNTechnique's sampler2D inputs
- Why does my scene camera only show black screen when I use SCNTechnique?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?