Post: I've discussed some of the details of this midrash of parashat Matot in the past (here and here), and considered what could be the spark
According to this midrash, related as well by Rashi:
|6. Moses sent them the thousand from each tribe to the army, them along with Phinehas the son of Eleazar the kohen to the army, with the sacred utensils and the trumpets for sounding in his possession.||ו. וַיִּשְׁלַח אֹתָם מֹשֶׁה אֶלֶף לַמַּטֶּה לַצָּבָא אֹתָם וְאֶת פִּינְחָס בֶּן אֶלְעָזָר הַכֹּהֵן לַצָּבָא וּכְלֵי הַקֹּדֶשׁ וַחֲצֹצְרוֹת הַתְּרוּעָה בְּיָדוֹ:|
|the sacred utensils: The holy Ark (Sifrei Mattoth 34, Num. Rabbah 22:4) and the golden showplate (Mid. Aggadah), since Balaam was with them and through sorcery was able to make the Midianite kings fly, and he flew along with them, he [Phinehas] showed them the showplate on which God’s Name was engraved, and they fell down [to earth]. For this reason it says, concerning the Midianite kings, “upon their slain” (verse 8), for they fell from the air on top of those slain. Likewise, it says in the book of Joshua (13:22) in connection with Balaam,“upon (sic) their slain.” - [Mid. Tanchuma Mattoth 4]||וכלי הקדש: זה הארון והציץ. שהיה בלעם עמהם ומפריח מלכי מדין בכשפים, והוא עצמו פורח עמהם. הראה להם את הציץ, שהשם חקוק בו, והם נופלים, לכך נאמר על חלליהם במלכי מדין, שנופלים על החללים מן האויר, וכן בבלעם כתיב (יהושע יג, כב) אל חלליהם:|
While this is quite the thrilling tale, and while it has good hand-holds in the Biblical text itself, in this instance I would suspect that there is an allegorical meaning in play.
That message is that despite the various seemingly magical implements listed in this pasuk, this is not kishuf, but rather Hashem's direct protection of His holy nation.
One might think, incorrectly, that these items were purely magical means to win a war. Thus, in regard to the trumpets (as I discussed here, in Ibn Caspi and the magic trumpets, the pasuk says, in Behaaloscha:
|9. If you go to war in your land against an adversary that oppresses you, you shall blow a teruah with the trumpets and be remembered before the Lord your God, and thus be saved from your enemies.||ט. וְכִי תָבֹאוּ מִלְחָמָה בְּאַרְצְכֶם עַל הַצַּר הַצֹּרֵר אֶתְכֶם וַהֲרֵעֹתֶם בַּחֲצֹצְרוֹת וְנִזְכַּרְתֶּם לִפְנֵי יְ־הֹוָ־ה אֱלֹהֵיכֶם וְנוֹשַׁעְתֶּם מֵאֹיְבֵיכֶם:|
This seems to indicate that these trumpets have "magical" power, such that despite merit, tefillot, etc., Hashem is waiting for these trumpets, which "remind" Him and then you win the war.
So too the aron, if that is the klei hakodesh. From Shmuel Aleph, perek daled:
It is easy to fall into that trap and think that the aron is a sort of mystical weapon of war, which ensures victory. Compare to Bilaam who also was used as a mystical weapon of war, making use of kesamim, and where his curse could purportedly cause a nation to win or lose a battle.
And perhaps it is not a trap in thinking, but these things bring about Hashem's remembrance or His Divine focus. And so the purpose of the midrash is to cast the klei kodesh as our versions of the weapon.
I think the message, though, is the opposite. As Bilaam said in parashat Balak:
|כג כִּי לֹא-נַחַשׁ בְּיַעֲקֹב, וְלֹא-קֶסֶם בְּיִשְׂרָאֵל; כָּעֵת, יֵאָמֵר לְיַעֲקֹב וּלְיִשְׂרָאֵל, מַה-פָּעַל, אֵל.||23 For there is no enchantment with Jacob, neither is there any divination with Israel; now is it said of Jacob and of Israel: 'What hath God wrought!'|
And so, the Midianites attempted to use kesamim and keshafim of various sorts to win the war. Pinchas wearing the bigdei kehuna was representative of the Divine presence, of Hashem championing their cause, and that the Israelite people had been chosen. When the Midianim tried to use magic to win, or to escape their fate, by making use of Bilaam's powers, Pinchas showed them the tzitz, either because it had Hashem's name engraved on it or because it has Kodesh LaHashem engraved on it, where the former would represent Hashem's control and the latter would represent Israelite chosenness. This caused them to plummet, not because the tzitz is magical, but because their plummeting represents the sudden failure of their magic, in catastrophic circumstances.
Whether or not this midrash was also intended literally, it would be a shame to focus on only the literal level (and possibly reject that), and miss out on its full import.