Figures in higher education feel double standards are being used to delegitimise work of minority groups