This sidra is called Emor, because it is the first unique word to occur. We don’t call it parshas Vayomer, and we don’t call other parshiyos Vayedabar. I’ve heard it referred to as parshas Emor el HaKohanim, but really, Emor suffices.
Why the strange language of Emor? In the Biur, by the Baal Nesivos Hashalom (that oisvorf*), we see the following explanation:
“(1) Vayomer Hashem El Moshe: We explained above (parasha 1:1) [where the pasuk contains both דבור and אמירה, since it reads וַיִּקְרָא, אֶל-מֹשֶׁה; וַיְדַבֵּר ה אֵלָיו, מֵאֹהֶל מוֹעֵד לֵאמֹר] that אמירה encompasses broad matters and short [precise] matters, while דבור always pertains to broad matters.And when אמירה is juxtaposed to דבור this teaches that He spoke [דבר] broadly, and also commanded him [Moshe] to speak [לאמר] to the nation in the [specific] language which follows in the section. And so too here, Vayomer Hashem el Moshe, according to the language written in the section. So too Emor el Hakohanim Benei Aharon [the next phrase in the pasuk], with that specific language which I tell [אומר] you.
And the reason that both דבור and אמירה are not stated here as well, while without doubt He spoke [דבר] with him and instructed him all of the halachot [broadly, even those not stated in the section] prophetically, I will explain with the help of Hashem (in pasuk 24).”
Emor starts in perek 21, and pasuk 24 is the last pasuk of that perek. There are two sections in this perek. The first (pasuk 1-15) is an instruction from Moshe to the Kohanim the sons of Aharon specifically, as we see above in pasuk 21:1. The contents of that instruction pertain to defilement, conduct, and marriage. The second section (starting at Sheni, after a setuma break, pasuk 16 - 23) is an instruction from Moshe to Aharon, and discusses blemishes:
Finally, at the end of the perek stands pasuk 24. The instruction of Vaydabar has to relate to what is above, because what follows in Vayikra 22:1 is most certainly a brand new section.
Here, in pasuk 24, we are told that Moshe spoke to three groups - Aharon, Aharon’s sons, and the Bnei Yisrael.
The Beur explains:
“(24) Vaydaber Moshe el Aharon etc.: I think this refers back to the two preceding sections. For in the first [1-15] Aharon and Yisrael are not mentioned, and in the second, his [Aharon’s] sons and the Bnei Yisrael are not mentioned. And further, in the first דבור is not mentioned, that he explained for them the matters in with a clear, comprehensive explanation [באר היטב]. Therefore, it closes with this pasuk in which is stated דבור, and in which is mentioned Aharon, his sons, and all of Yisrael, so distribute what is written in one to the other.
And Rashi za”l explains [the newly introduced here in pasuk 24] el Bnei Yisrael, that the [Israelite] Beit Din should warn the kohanim. And so it is in Torat Kohanim.”
* added at yaak's request.