Michael Tänzer

Michael Tänzer, Sebastian Ruder, Marek Rei

State-of-the-art pre-trained language models have been shown to memorise facts and perform well with limited amounts of training data. To gain a better understanding of how these models learn, we study their generalisation and memorisation capabilities in noisy and low-resource scenarios. We find that the training of these models is almost unaffected by label noise and that it is possible to reach near-optimal results even on extremely noisy datasets. However, our experiments also show that they mainly learn from high-frequency patterns and largely fail when tested on low-resource tasks such as few-shot learning and rare entity recognition. To mitigate such limitations, we propose an extension based on prototypical networks that improves performance in low-resource named entity recognition tasks.

Michael Tänzer

Often, security is considered in an advanced stage of the implementation of a system, rather than integrating it into the system design. This leads to less secure systems, as the security mechanisms are only applied as an afterthought and therefore do not integrate well with the rest of the design. Also, several statistics about discovered vulnerabilities in existing systems suggest, that most of the vulnerabilities of a system are not caused by errors in the cryptographic primitives, but in other parts of the implementation. So integrating security concerns early in the design process seems a promising approach for increasing the security of the resulting system. This work evaluates how the choice of the architectural style affects the security of the resulting system. The evaluation is done on the example of an existing certification authority (CA). The requirements for the system are gathered and multiple designs according to different architectural styles are drafted and evaluated using a risk evaluation method. Then the evaluated designs are compared to find out whether there are significant differences.