A while ago I started to build something similar, but I didn't get much farther than choosing a name for the project (https://github.com/mk12/loopy). I want to design a language that can specify patterns precisely and unambiguously, and then visualize them in a physically accurate way. For basic 2D patterns I was thinking I could render each stitch based on it and its neighbors (3x3 grid of K/P). But that wouldn't be that useful. What would really be neat is to use a physics engine to simulate the yarn in 3D. For example the simulations would show stockinette curling and ribbing stretching without that being hardcoded anywhere. But I think it's too hard and I have no idea where to start with that.