16  Static & Dynamic Models

Take-home Message — Thought conversion 1:

“Soft” communication shares many qualities of “hard” machine learning. “Hard” deep learning is ground-breaking because of, not despite of its “soft” qualities.

Imagine the parameters to a model

\[y_i = \beta_{0} + \beta_{1} x_{i1} + \cdots + \beta_{p} x_{ip} + \varepsilon_i\]

16.1 3. Two extremes

Static/Finite Disciplines Dynamic/Infinite Disciplines
Classical models (goal/object of discipline) Seek universal truths, e.g. In the form of natural laws & robust theories Seeking unified understanding (universal truths are rarer and more difficult to develop)
Techniques Closed, controlled environments, where the entirety of a system may be described Even with closed environments, the infinite complexity of the real world is difficult to capture. At most we can define broad trends and generalizations.
Certainty Concrete thinking Probabilistic thinking
Approach Analytical Thinking Systems thinking
Logic Deductive reasoning Inductive reasoning
Disciplines Classical physics & chemistry and to a point molecular genetics Systems Biology (ecology, earth science) & psychology

16.2 4. A Scenario

Now imagine the simple actions of two people talking to each other. We can imagine two models are at play for each person depending on what they are doing:

Talking Listening
Model Describes an encoder transmiting information. Describes a decoder receiving information.
Input Thought Speech
Output Speech Thought

TalkingDescribes a model where an encoder transmitsInput in thoughtoutput is speech ListeningDescribes a model where a decoder receivesinput is speechoutput is thought Input, model and output are Interdependent, constrained contextual. Dynamic/Infinite models are also present in project management, decision making & other areas outside of actual technical work.

16.3 5. In Classical Definitions

Static/Finite models in classically soft disciplines Dynamic/Infinite models in classically hard disciplines

  • cf. cases where e.g. biology or psychology try to trace a direct path between input and output. This is possible, (but it appears quite rare).

  • Classical Mendialian genetics, which is actually hite rare.

  • Diagnostic markers or a disearst (biomarkers, genetic variants) which are also rare.

  • Deep learning is a great example of the development of dynamic/infinite models combined with classical hard disciplines, like maths.

  • Early AI attempted to predict all possible outcomes of a system, which turned out to be an imposible task.

  • ML algorithims defines the model and the hyperparameters then tried to use the right model on the data

  • Then DL let the model redefine according to the data

  • Upcoming:

    • Foundational models

    • Data-centric models, instead of modifying models, get more data and clean up the data to get more information. (see Andrew Ng.)