Architectural Constraints on Representational Ontology in Neural Audio Models

2024 – Present, Yale University

​

Project Overview

Through their mathematics, modern neural architectures define, in advance, which internal representations they can reliably stabilize. This project investigates how architectural structure—particularly convolutional operator design, compression, and hierarchical abstraction—constrains the representational ontology of neural audio models.

I distinguish between a model’s latent ontology (the representational affordances implied by its architecture) and its operational ontology (the internal structures that actually stabilize under training). While training determines which features emerge, architecture determines what kinds of features are even capable of becoming independently controllable internal variables. This distinction allows us to separate limitations imposed by data from those imposed by model design.

Methodologically, this work combines mathematical analysis of operator structure with intervention-based probing of trained models. Across a family of neural audio models, I show that compression and downsampling structurally induce feature entanglement, creating a representational ceiling independent of scale. Certain oscillatory features, despite being foundational to human semantic descriptions of sound, rarely stabilize as monosemantic internal variables except in sparse edge cases. Increasing model size improves the coverage and reuse of internal directions, but it does not eliminate the entanglement pressure imposed by architectural constraints.

Taken together, this research argues that steerability and safety are not purely matters of training dynamics, data, or scale. They are also bounded by the representational affordances embedded in architectural design.