My manager expects instant replies even during flex hours.

If I shift my schedule earlier or later, I still get messages asking why I'm not responding 'immediately.' So what's the point of flex hours?