I don't think there has to be. Many libraries for example overkill it and make everything a layer when it should be a layer with a hyper parameter. A good example of this is drop out. Why not make it a knob rather than a layer?
I think people should be given high level primitives like "layers", allow them to make their own where necessary, but allow the defaults to be:
Layer with x (dropout,momentum,..) trained by optimization algo: LBFGS,Hessian Free,..
This allows people to experiment with different configurations without having to dive deep to achieve some basic problems.
Relevant to julia: it's a great language and what I wish production code could look like (while being fast!)
Like rust, it's in a pretty alpha state right now. I'm watching the language heavily though.
I think people should be given high level primitives like "layers", allow them to make their own where necessary, but allow the defaults to be: Layer with x (dropout,momentum,..) trained by optimization algo: LBFGS,Hessian Free,.. This allows people to experiment with different configurations without having to dive deep to achieve some basic problems.
Relevant to julia: it's a great language and what I wish production code could look like (while being fast!)
Like rust, it's in a pretty alpha state right now. I'm watching the language heavily though.