Sprints with Contractors

Hey all,

I'm relatively new to my company and am facing a challenge which I can hopefully get some perspective on. 

When I first started I inherited two separate product releases which were both partly done and way behind schedule (were built via waterfall).  First step I did was attempt to break each product into smaller iterations and order them in a way that we could focus one at a time.  My backlog resulted in something like;

Project 1 - Feature Area A

Project 1 - Feature Area B

Project 1 - Feature Area C

Project 1 - Feature Area D

Project 2 - Feature Area A

Project 2 - Feature Area B

Project 2 - Feature Area C

We didn't get too far when my company threw a new initiative at me that I broke into 3 parts.  I'll just call them projects X, Y and Z.  We did a much better job at delivering project X using core agile principals but I had to hire a contractor who has a skill set that nobody else on my team had.  The contract is signed for the duration I estimated for completing X, Y and Z.  However, when you look at my backlog now I still need to get projects 1 and 2 out before Y and Z.

Given that my one contractor has a unique skill set that I'm paying by the hour and does not have the skills nor background to add any real velocity to project 1 and 2, how do I convince my team to allow my one person to press forward with Y and Z who can certainly deliver independent of the rest of the team?  Am I being a poor agile manager?  Or should I just move projects Y and Z as the top priorities due to budget implications even if it doesn't truely reflect business/customer need?