Nested Parentheses

We investigate if, given a simple symbol mask- ing strategy, self-attention models are capable of learning nested structures and generalise over their depth. We do so in the simplest set- …...