To understand how people perceive these blends, we put together a survey and asked people to rate 88 blends. We gave them an example of a blend without telling them what words it’s made from and asked them to answer the following questions about it:
1. Do you understand what it means? If they said no, they could skip straight to the next word.
2. Understandability: Is it easy to understand what words make up this blend?
3. Naturalness: Does this combination of words sound natural to you?
We knew in advance that it’s not always easy to decide what the difference between understandability and naturalness is; presumably, difficult to understand blends are also unnatural. We’re most interested in the examples where those measures don’t line up: which blends are easy to understand but unnatural (and vice versa)? We’ll look at the difference between those two measures below. For convenience, when giving average ratings we convert the 5-point Likert scale participants used (Terrible, Poor, Fair, Good, Excellent) to a 1-5 scale. (Note for the quantitatively minded: in the actual modeling we perform, we don’t do this conversion and do not make the interval assumption or any normality assumptions regarding the data.)