I wonder if the inductive bias (or the prior structure) changes or evolves when it experiences the world ! Or does it stay rigid invariant throughout training ? If the prior structure really guides learning then how can it stay rigid when the experience itself is evolving ? Finite evidence cannot determine its own extrapolation but the conditions that are necessary for this extrapolation, are those universal ?
I wonder if the inductive bias (or the prior structure) changes or evolves when it experiences the world ! Or does it stay rigid invariant throughout training ? If the prior structure really guides learning then how can it stay rigid when the experience itself is evolving ? Finite evidence cannot determine its own extrapolation but the conditions that are necessary for this extrapolation, are those universal ?