Sorting in Custom Reports Doesn't seem to work

It doesn't seem like the Sort Order works at all for Custom Reports. I have created a custom report from the Line Item Detail (Unit Price and Total Price) report I only wanted to remove the description and add a part number, but the sorting is no longer alphabetical as soon as I create a Custom Report based on that report. When I add "Sorting" to the custom report, the sorting does not work at all.