GAMMT: Generative Ambiguity Modeling Using Multiple Transformers
This addresses the challenge of uncertain data generation processes in sequential modeling, but it is incremental as it builds on existing transformer architectures without proven impact.
The paper tackles the problem of modeling ambiguous sequential data by introducing GAMMT, a generative model using multiple parallel transformers linked by a selection mechanism to approximate ambiguous probabilities, but it lacks experimental validation and concrete results.
We introduce a novel model called GAMMT (Generative Ambiguity Models using Multiple Transformers) for sequential data that is based on sets of probabilities. Unlike conventional models, our approach acknowledges that the data generation process of a sequence is not deterministic, but rather ambiguous and influenced by a set of probabilities. To capture this ambiguity, GAMMT employs multiple parallel transformers that are linked by a selection mechanism, allowing for the approximation of ambiguous probabilities. The generative nature of our approach also enables multiple representations of input tokens and sequences. While our models have not yet undergone experimental validation, we believe that our model has great potential to achieve high quality and diversity in modeling sequences with uncertain data generation processes.