Currently the priority button is only available in task mode. But the related functionality can also be found in the event/task dialog and in calendar mode. We should make the toolbarbutton available there as well. However I would recommend to not display it by default and only make it available via toolbar customization in order to not bloat the toolbars. Christian, I would be interested in your opinion on this.
For the event/task dialog a priority toolbar button is already requested with Bug 362933.
With the redesigned toolbars, this has essentially become a dupe of bug 362933.