A Transformer-based Approach for Source Code Summarization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 460–464 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 460 A Three-Parameter Rank-Frequency Relation in Natural Languages Chenchen Ding, Masao Utiyama, Eiichiro Sumita Advanced Translation Technology Laboratory, Advanced Speech Translation Research and Development Promotion Center, National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan {chenchen.ding, mutiyama, eiichiro.sumita}@nict.go.jp Abstract We present that, the rank-frequency relation in textual data follows f ∝ r−α(r+γ)−β, where f is the token frequency and r is the rank by frequency, with (α, β, γ) as parameters. The formulation is derived based on the empirical observation that d2(x+y)/dx2 is a typical im- pulse function, where (x, y) = (log r, log f). The formulation is the power law when β = 0 and the Zipf–Mandelbrot law when α = 0. We illustrate that α is related to the analytic fea- tures of syntax and β + γ to those of morphol- ogy in natural languages from an investigation of multilingual corpora. 1 Introduction Zipf’s law (Zipf, 1935, 1949) is an empirical law to formulate the rank-frequency (r-f) relation in physical and social phenomena. Linguistically, Zipf’s law can be observed on the distribution of words in corpora of natural languages, where the frequency (f) of words is inversely proportional to its rank (r) by frequency; that is, f ∝ r−1. Zipf’s law is a special form of a general power law, that is, f ∝ r−α, with α = 1. The Zipf’s/power law is usually examined un- der a log-log plot of rank and frequency, where the data points lie on a straight line. The sim- ple proportionality of the Zipf’s/power law can be observed on randomly generated textual data (Li, 1992) and it only roughly depicts the r-f relation in real textual data. A two-parameter generalization of the Zipf’s/power law is the Zipf-Mandelbrot law, where f ∝ (r + β)−α (Mandelbrot, 1965). Li et al. (2010) considered the reversed rank of rmax+1−r, where rmax is the maximum of rank- ing index, and proposed a two-parameter formula- tion of f ∝ r−α(rmax + 1 − r)β. As a straightforward observation, the coeffi- cients of proportionality should be distinguished for common and rear words (Powers, 1998; Li 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 Figure 1: Rank-frequency plots on English words (left) and Chinese characters (right). The x- and y-axes are log10 r and log10 f, respectively. The gray curves are the proposed formulation under logarithm: y = C − αx − β log10(10x + 10γ), where C is a constant. The dashed lines are the asymptotes of C − (αx + βγ) and C−(α+β)x. (α, β, γ) is (0.93, 2.04, 3.82) for English words and (0.59, 32.31, 4.42) for Chinese characters. et al., 2010). Therefore, an extension of the origi- nal Zipf’s/power law …

Content truncated to 500 words