This is partly because you have had a day of exhausting bug-bashing. You can’t see it happening, but your body is extremely busy defending you against the effects of the virual infection and mending damage that has been done. It’s very demanding work, and you won’t have any energy left over for working or running around, or sometimes even reading or talking.
The other reason for the worsening of symptoms in the evening is that levels of inflammatory chemicals naturally rise during the day and are highest in the evening. As colds or flu infections involve inflammation too, the evening combination can exacerbate symptoms.