We use a mixture of LLMs (e.g. to identify the important variables to trace), language servers (to understand where a variable is originally defined and where it's used).

We may go back to code visualization for editors, but after testing it with a lot of engineers, we found that it was too confusing for code reviews. The main issue was just how alien the interface is!

> 1. Allow me to step through the code execution paths that have been modified in the pull request, based on the tests that have been modified.

Not sure if I fully grasp this! We tried to kind of do this in previous iterations (show call graphs all at once) and it gets messy very fast. Could you elaborate on this point in particular?

Sure - imagine my PR adds one new test which test one new function.

Starting from the test, allow me to step through the program execution, just like a debugger, to observe variables, surrounding code, and the complete file.

If you read only the covered lines of code in a linear way, you'd miss the refactoring opportunities because you aren't looking at the rest of the file.

Yes! I personally iterate without understanding the code too much (since it'll change), and fully assess what Claude Code (in my case) has done after I finished a piece of work.

Thanks for the feedback!

Not sure if I fully grasp what you mean by dog whistling, but at the end of the day, like another commenter said, Haystack is also pretty helpful for when you're done experimenting with a piece of work and need to see what an AI has generated.

Yes! We haven't wholly given up on that idea by the way ;)

>If AI writes all of the code, we will need to max out humans’ ability to do architecture and systems design.

Strongly agree with this. There are demos that you're able to try, by the way!

Yeah I did this too as an engineer!

I think this is a valid part of the "crafting PR" skill that's under appreciated, and part of the goal of Haystack here is to make that part of PR craft effortless.

The effort is part of what makes the outcome better, though.

Sorry to clarify: you hit "back" after traversing from the summary but it doesn't go back? If so, that's a bug!

Or do you mean that doing the browser navigation of "back" should bring you to the summary (initial page)?

Sorry for the confusion! Missing browser navigation is the problem. The virtual back buttons you put in the top left are working as I'd expect browser nav to do. I keep trying to go back, it would feel so natural.

Ahah! Thanks for the feedback.

> you likely have a bigger organizational issue

Could you expound on this? In my experience as a software engineer, a pull request could fall into one of two buckets (assuming it's not trivial):

1. The PR is not organized by the author so it's skimmed and not fully understood because it's so hard to follow along

2. The PR author puts a lot of time into organizing the pull request (crafting each commit, trying to build a narrative, etc.) and the review is thorough, but still not easy

I think organization helps the 1st case and obviates the need for the author to spend so much time crafting the PR in the 2nd case (and eliminates messy updates that need to be carefully slotted in).

Curious to hear how y'all handle pull requests!

This is where I feel like we've solved a third-order problem. If you're sorting all PRs into those two buckets then you should probably take a step back and redefine what a PR is for your organization, as both 1 and 2 make the assumption that the PR is too big to review in a single sit down or that the author didn't put in enough effort to craft their PR. Both of these should just be rejected outright in favor of doing things in a smaller more manageable way, instead of having an AI sort through something that a human should have started with. Obviously this is more of an ideal situation and a lot of companies don't work on the ideal which is why I think your product will find good use because companies don't want to invest in slowing down, only going faster.

Interesting. At my previous company there was a debate about smaller PRs vs bigger PRs and the end conclusion was that there are tradeoffs in being able to deal with 2-5 bite-sized PRs vs one large PR. The largest one being that it's hard to grasp the totality of the pull request and how the different PRs work together.

> companies don't want to invest in slowing down, only going faster.

I do think this is the way things are going to go moving forward, for better or for worse!

> you should probably take a step back and redefine what a PR is for your organization

I agree with this wholeheartedly if you are in a role that allows you to redefine what a PR is. In almost every organization that I've worked for, the PR is defined several levels above my pay grade and suggesting changes/updates/etc is usually seen as complaining.