Dhimmis, or people of the book, made up the greater portion of the population of the Umayyad empire. This name originally applied to Christians and Jews, who shared the holy book with the Muslims. These people were allowed to worship as they pleased.