fix(thrift): bound skip() recursion depth to prevent stack overflow#40
fix(thrift): bound skip() recursion depth to prevent stack overflow#40moshap-firebolt wants to merge 1 commit into
Conversation
lorenzhs
left a comment
There was a problem hiding this comment.
a) this change looks rather different than the removal, why? https://24d7106e4280c32d65165cd052b8309fd06478b7
b) what's with the random base branch? Why are we not using release-24.0? Please fix
|
merged the base branch PR |
822adc2 to
00e2a70
Compare
…erflow Commit apache#86 ("bare bones Thrift") removed the recursion-depth tracker, leaving the generic skip() helper — and the generated Parquet deserializer, which routes every unknown field through skip(ftype) — able to recurse without limit. A crafted Parquet footer with deeply nested unknown Thrift fields overflows the stack (SIGSEGV, uncatchable by the reader's try/catch). Found by the griffin_parquet_column storage fuzzer. Restore the mechanism that apache#86 dropped rather than reimplement it: make TInputRecursionTracker / TOutputRecursionTracker real (templated on the protocol, since there is no longer a common TProtocol base) and give TCompactProtocolT the input/output depth counters + limit (from the transport's TConfiguration, default 64) they drive, throwing TProtocolException::DEPTH_LIMIT when exceeded. skip() reverts to its original form with a TInputRecursionTracker at the top; the tracker instantiations already present in the generated parquet_types.tcc are now effective again too.
00e2a70 to
8b42b44
Compare
|
Thanks @lorenzhs — both addressed: (a) Good call. Rewrote it to restore the mechanism apache#86 removed rather than reimplement it: (b) Already handled by your rebase onto Verified with a clang-22 + ASAN repro (incl. the generated-code-style CTAD call site): deep nested input throws |
Background
The storage fuzzer (
griffin_parquet_column) found a stack overflow parsing a crafted Parquet footer: packdb run 29895460517.This vendored Thrift copy stubbed Apache Thrift's recursion-depth trackers as no-ops (see the comment above
TInputRecursionTracker), so the genericapache::thrift::protocol::skip()helper recurses without limit. The generated Parquet deserializer routes every unknown field throughiprot->skip(ftype)(229 sites), so a deeply nested unknown struct/list/set/map field drives unbounded recursion and overflows the stack. A stack overflow is a signal, not a C++ exception, so the reader'stry/catcharoundParquetFileReader::Opencannot contain it.Summary
Restore the upstream recursion guard in
skip(): thread adepth_limit(defaulting toTConfiguration::DEFAULT_RECURSION_DEPTH= 64) and throwTProtocolException::DEPTH_LIMITwhen it is exhausted. The two-argumentskip(prot, type)overload is preserved for existing callers.Test Plan
skip(T_STRUCT):AddressSanitizer: stack-overflowin theskiprecursion (matches the CI stack trace).TProtocolExceptiontype=6 (DEPTH_LIMIT); shallow/valid inputs parse unchanged.🤖 Generated with Claude Code