[Feature] MSAA (Hardware Multisample Anti-Aliasing)#307
Open
keithwill wants to merge 3 commits into
Open
Conversation
The grab-pass temporary RT was always allocated as Color4b, even when the scene color it copies from is HDR (Short4). glBlitFramebuffer permits format conversion when the read framebuffer is single-sampled, so this silently truncated HDR grab/refraction passes to 8 bits per channel rather than erroring. Allocate the grab RT with the source RT's own format. Dimensions already match on both sides, so the blit stays legal.
Prowl's only anti-aliasing was post-process (FXAA/SMAA/TAA), which reconstructs edges from an already-aliased image. MSAA antialiases at rasterization time instead, so it cannot be an ImageEffect -- it has to live in the render targets. Adds Camera.MSAA (None/X2/X4/X8, default None) which makes the forward geometry passes rasterize into a multisampled color+depth target, resolved back to an ordinary single-sampled texture before anything samples it. Everything from PostProcess onward -- effects, gizmos, the final blit -- is untouched and behaves exactly as it does with MSAA off. Being a public field, it surfaces in the inspector via the reflective property grid with no editor code. Because the resolve goes through glBlitFramebuffer and multisampled content is never sampled in a shader, no GLSL changes and no sampler2DMS are needed. Notes on the less obvious parts: - The prepass stays single-sampled. It feeds _CameraDepthTexture/Normals/Motion, which GTAO/SSR/TAA read as plain sampler2D. - The prepass -> colorRT depth blit is skipped under MSAA. It only ever provided opaque early-Z: the skybox is ZTest Off/ZWrite Off and is overdrawn regardless, and opaques regenerate correct per-sample depth themselves (RasterizerState defaults to ZWrite On/LEqual). The replacement depth clear is emitted outside the ClearFlags switch because the Depth and Nothing branches never clear, and colorRT is pooled, so they would otherwise inherit stale depth. - The Depth/Nothing branches copy the target with a fullscreen quad rather than a framebuffer blit: RenderScale makes target and colorRT different sizes, and a scaling blit into a multisampled framebuffer is INVALID_OPERATION. - Blit.shader gains a Blend Off "Copy" pass. The existing pass is Blend Alpha, which would blend rather than replace when copying into a multisampled target, and HDR scene color carries no alpha=1 guarantee. - GTAO/SSR read and write SceneColor, so under MSAA the target is resolved for them and copied back, leaving the multisampled depth buffer live so transparents still rasterize with real coverage. Only paid when those effects are present. - Multisampled textures reject all sampler state (INVALID_ENUM), so the Texture base constructor skips its default wrap/filter setup for them. - RenderTexture's pool key carries SampleCount in both the key struct and the rebuild in ReleaseTemporaryRT; missing either would silently alias multisampled and single-sampled targets. Verified on-device across a matrix of 16 configurations (sample counts, HDR, all four ClearFlags, RenderScale 0.5/2.0, GTAO/SSR, post-process stacks, and MSAA toggling for pool reuse) with a GL debug context: no GL errors. The check was falsified by temporarily removing the multisample sampler-state guard, which correctly reported GL_INVALID_ENUM on GL_TEXTURE_2D_MULTISAMPLE.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds MSAA as a per-camera rendering option, alongside the existing post-process AA effects (FXAA/SMAA/TAA). It gives Prowl its first anti-aliaser that works at rasterization time rather than reconstructing edges from an already-aliased image.
MSAA
FXAA, SMAA, and TAA all run at the end of the frame on a finished image: the geometry has already been rasterized with one sample per pixel, so the edge information is gone and has to be inferred. MSAA instead rasterizes coverage and depth at multiple samples per pixel and resolves them, so geometric edges are correct rather than reconstructed — no blurring of fine detail (FXAA), no ghosting or history buffers (TAA). The trade is memory and bandwidth instead of a fullscreen pass, and it only helps geometric aliasing, not shading or texture aliasing, so it composes with the post-process effects rather than replacing them.
Because it lives in the render targets, MSAA cannot be an
ImageEffect— it has to be part of the pipeline itself.How it was implemented
Camera.MSAA(None/X2/X4/X8, defaultNone, clamped toGraphics.MaxSamples) makes the forward geometry passes rasterize into a multisampled color+depth target, which is resolved back to an ordinary single-sampled texture before anything samples it. Everything fromPostProcessonward — effects, gizmos, the final blit — is untouched and behaves exactly as it does with MSAA off.Camera.Targetstays single-sampled; MSAA is internal to the pipeline.The resolve goes through
glBlitFramebufferand multisampled content is never sampled in a shader, so no GLSL changes and nosampler2DMSare needed. Being a public field,MSAAsurfaces in the inspector through the reflective property grid with no editor code.Notes on the less obvious parts
_CameraDepthTexture/Normals/Motion, which GTAO/SSR/TAA read as plainsampler2D.ZTest Off/ZWrite Offand is overdrawn regardless, and opaques regenerate correct per-sample depth themselves. Its replacement depth clear is emitted outside theClearFlagsswitch, because theDepthandNothingbranches never clear andcolorRTis pooled — they would otherwise inherit stale depth.Depth/Nothingbranches copy the target with a fullscreen quad rather than a framebuffer blit:RenderScalemakes target andcolorRTdifferent sizes, and a scaling blit into a multisampled framebuffer isINVALID_OPERATION.Blit.shadergains aBlend Off"Copy" pass. The existing pass isBlend Alpha, which would blend rather than replace when copying into a multisampled target, and HDR scene color carries noalpha = 1guarantee.INVALID_ENUM), so theTexturebase constructor skips its default wrap/filter setup for them.RenderTexture's pool key carriesSampleCountin both the key struct and the rebuild inReleaseTemporaryRT; missing either would silently alias multisampled and single-sampled targets.Included fix: grab-pass RT format
Also included is a small independent fix that MSAA would have turned into a hard error. The grab-pass temporary RT was always allocated as
Color4beven when the scene color it copies from is HDR (Short4).glBlitFramebufferpermits format conversion from a single-sampled read framebuffer, so this silently truncated HDR grab/refraction passes to 8 bits per channel instead of erroring. The grab RT now takes the source RT's own format.Usage
Testing
Added
MSAATests— headless tests covering the enum's literal sample counts, the camera default, multisampledRenderTexture/Texture2Dconstruction and its invalid-argument cases, pool behavior (no aliasing across sample counts, reuse within one), theBlend Offcopy pass inBlit.shader, and the guards on blitting from a multisampled source and resolving mismatched sizes.Verified on-device across a matrix of 16 configurations (sample counts, HDR, all four
ClearFlags,RenderScale0.5/2.0, GTAO/SSR, post-process stacks, and MSAA toggling for pool reuse) under a GL debug context: no GL errors. The check was falsified by temporarily removing the multisample sampler-state guard, which correctly reportedGL_INVALID_ENUMonGL_TEXTURE_2D_MULTISAMPLE.