do you mean accessing data outside of your app's framebuffer? or just accessing neighboring pixels during a shader pass, because those are _very_ different things. GPU MMUs mean that you cant access a buffer that doesn't belong to your app that's it, its not about restricting pixel access within your own buffers
are succinct data structures good for "dynamic" applications? ie modifying the data often, last time i looked into this field it seems that all the popular structures were very static in nature and had costly updates
Maybe there are more modern, advanced data structures / techniques, but yes, my understanding is that this is a common trade off (having your data structure be more static, with offline updates).