Add the path-stitching algorithm #9
Conversation
rewinfrey
left a comment
rewinfrey
left a comment
There was a problem hiding this comment.
I think this looks 👍 and really appreciate all the 📝 ! I focused mostly on the append_partial_path code path to better internalize the handling of symbol and scope stacks and the symbol and scope bindings. I left a couple questions and one small request for a bit of additional 📝 . Great work 😄
| ) -> Handle<PartialPath> { | ||
| let start_node = path.start_node; | ||
| let symbol_stack_precondition = path.symbol_stack_precondition; | ||
| let handle = self.partial_paths.add(path); |
There was a problem hiding this comment.
Is there the possibility of loading partial paths concurrently? Or is that something we'd should explicitly not allow based on the add operation? It doesn't look like that would be thread safe.
There was a problem hiding this comment.
It's purposefully not safe to load partial paths concurrently, you have to have mut access to the PartialPaths arena. Over in the c-api-* branches I'm working on C wrappers that would let you pass in batches of objects to add to the stack graph. So the Go code will be able to load in a bunch of partial paths in worker goroutines, and then hand them off to a single C call to add them to the Rust code's internal state. For instance:
Lines 452 to 467 in dd3bd54
| self.symbols.push_front(&mut db.symbol_stack_keys, symbol); | ||
| let handle = self.back_handle(); | ||
| db.symbol_stack_key_cache.insert(cache_key, handle); | ||
| } |
There was a problem hiding this comment.
I'm used to thinking about stacks in terms of "top" and "bottom" of the stack, and have been translating "front" -> "top" and "back" -> "bottom". Is that the right idea?
There was a problem hiding this comment.
Yes, I've been using front and back to describe how they're typically written in our CLI output, where the top of the stack is on the left (i.e. the front of the list of symbols). e.g. in cheese.ints.one, the stack consists of 5 symbols: cheese, ., ints, ., and one. cheese is the top or front of stack (and would be the thing popped off by any pop node we run across during pathfinding). one is bottom or back of stack.
|
|
||
| /// Pops a symbol from the back of this symbol stack key. | ||
| fn pop_back(&mut self, db: &Database) -> Option<Handle<Symbol>> { | ||
| self.symbols.pop_front(&db.symbol_stack_keys).copied() |
There was a problem hiding this comment.
Should this be self.symbols.pop_back instead of pop_front?
There was a problem hiding this comment.
Nope, we're using List under the covers, which only has push/pop_front. That means we're storing the content of the key in reverse order. That ends up not mattering since it's an internal detail; as long as we're consistent throughout the implementation of this type it will all work out. I'll add a clarifying comment to that effect.
This database maintains internal in-memory indexes to support all of the lookups that we might need to perform during the path-stitching algorithm. Most users will probably have some external storage layer holding partial paths, in which case you will be responsible for loading into the `Database` instance all of the partial paths that are valid extensions of some current path.
Partial paths can have variables in their preconditions and postconditions. When trying to append a partial path to a path, you "match" the partial path's precondition against the path's corresponding stack. Any variables in the precondition can "bind" parts of the path's stack. You then "apply" those bindings to the partial path's postcondition. Any variable references in the postcondition are substituted with the stack contents that were bound from the precondition. The result is the path's new stack after having performed the concatenation.
This is an implementation of the path stitching algorithm that's divided up into "phases". At the start of each phase, we process whatever (possibly incomplete) paths are currently in the queue. As we extend those paths with partial paths, we queue up the newly extended paths to be processed in the _next_ phase. This phasing approach means that the Database instance doesn't need to be pre-loaded with _all_ of the partial paths that we might ever need to process. Instead, you're notified of each path in the phase _before_ it will be processed, giving you a chance to add its extensions to the database before allowing the next phase to proceed.
This is an implementation of the path stitching algorithm that's divided up into "phases". At the start of each phase, we process whatever (possibly incomplete) paths are currently in the queue. As we extend those paths with partial paths, we queue up the newly extended paths to be processed in the next phase. This phasing approach means that the Database instance doesn't need to be pre-loaded with all of the partial paths that we might ever need to process. Instead, you're notified of each path in the phase before it will be processed, giving you a chance to add its extensions to the database before allowing the next phase to proceed.