Dear all,
Next week, we have the pleasure of hosting *Prof. Elad Hazan* (Princeton University) at our CS colloquium. The title and abstract of the talk appear below. For bio see here https://www.ehazan.com/bio/.
The seminar will be held on Monday, November 11th at 14:00. (Refreshments at 13:45)
*Location*: Romm C220, Rothberg Building.
Looking forward to seeing you,
Amir and Moshe
=============================== *Speaker: **Prof. Elad Hazan https://www.ehazan.com/bio/* (Princeton University) *Title*: Spectral Transformers *Abstract*: We'll discuss a new technique for sequence modeling for prediction tasks with long range dependencies and fast inference/generation. At the heart of the method is a new formulation for state space models (SSMs) based on learning linear dynamical systems with the spectral filtering algorithm. This gives rise to a novel sequence prediction architecture we call a spectral state space model. Spectral state space models have two primary advantages. First, they have provable robustness properties as their performance depends on neither the spectrum of the underlying dynamics nor the dimensionality of the problem. Second, these models are constructed with fixed convolutional filters that do not require learning while still outperforming SSMs in both theory and practice. The resulting models are evaluated on synthetic dynamical systems as well as long-range prediction tasks of various modalities. These evaluations support the theoretical benefits of spectral filtering for tasks requiring very long range memory.
The talk will be self-contained, but here is a link to more information about spectral transformers and filtering https://sites.google.com/view/gbrainprinceton/projects/spectral-transformers.
A reminder: this talk is tomorrow. See you there!
On Tue, Nov 5, 2024 at 9:45 AM Moshe Babaioff < moshe.babaioff@mail.huji.ac.il> wrote:
Dear all,
Next week, we have the pleasure of hosting *Prof. Elad Hazan* (Princeton University) at our CS colloquium. The title and abstract of the talk appear below. For bio see here https://www.ehazan.com/bio/.
The seminar will be held on Monday, November 11th at 14:00. (Refreshments at 13:45)
*Location*: Romm C220, Rothberg Building.
Looking forward to seeing you,
Amir and Moshe
=============================== *Speaker: **Prof. Elad Hazan https://www.ehazan.com/bio/* (Princeton University) *Title*: Spectral Transformers *Abstract*: We'll discuss a new technique for sequence modeling for prediction tasks with long range dependencies and fast inference/generation. At the heart of the method is a new formulation for state space models (SSMs) based on learning linear dynamical systems with the spectral filtering algorithm. This gives rise to a novel sequence prediction architecture we call a spectral state space model. Spectral state space models have two primary advantages. First, they have provable robustness properties as their performance depends on neither the spectrum of the underlying dynamics nor the dimensionality of the problem. Second, these models are constructed with fixed convolutional filters that do not require learning while still outperforming SSMs in both theory and practice. The resulting models are evaluated on synthetic dynamical systems as well as long-range prediction tasks of various modalities. These evaluations support the theoretical benefits of spectral filtering for tasks requiring very long range memory.
The talk will be self-contained, but here is a link to more information about spectral transformers and filtering https://sites.google.com/view/gbrainprinceton/projects/spectral-transformers.
-
Moshe Babaioff