It took me a while to figure this out but finally thanks to this forum post I did it. I think what was confusing me was you had to get the control and then set the label.
var affectedControl = Xrm.Page.ui.controls.get(fieldname);
var affectedAttribute = Xrm.Page.data.entity.attributes.get(fieldname);
affectedAttribute.setRequiredLevel(‘required’);
affectedControl.setLabel(‘New label’);