Show HN: I wrote an autodiff in C++ and implemented LeNet with it

  • using new for each node and value, combined with virtual dispatch tends to be a c++ anti-pattern. It looks like you are writing other languages in C++ syntax, motivated by promises of speed.

    The actual benefits of C++ come when you approach problems differently. This is a case where more exposure to C helps you avoid all the Java isms.

    Things to consider:

    - can you allocate memory for the whole system? - can you make types homogenous so they can fit in tight arrays (unions are common for nodes) - can you batch similar types - specially for auto diff/math can you represent operations as a stack instead of a tree?

    I am only bringing this up because you said your goal was to learn C++.

  • The actual C++/CUDA code is here:

    https://gitlab.com/mebassett/quixotic-learning/-/tree/master...

    about 1,000 LoC overall.