"...a complete please to use!"
— Cam Hart What They Say

How can I make the calendar start the week on Sunday?

You can go to Settings > General to make Week Starts On Sunday instead of Monday (the WordPress default).