EDIT: as /u/mfb- notes below, these numbers are off by a factor of 10 - all these percentages should be one decimal point to the right (e.g. 0.2% -> 0.02%).

Yes. Most of the CO2 was made by burning carbon-containing molecules using oxygen from the atmosphere, so each CO2 molecule roughly corresponds to one less O2 molecule.

But since CO2 is a small portion of the atmosphere's total, this doesn't make a big difference. Today, CO2 is about 420 ppm, or about 0.42%, of the atmosphere; prior to humans it was about 280 ppm (~0.28%). That's a huge difference in terms of how much CO2 there is; there's almost 50% more today than there was a couple centuries ago. But it implies a change of only about 0.14 percentage points in the oxygen amount.

Since oxygen is about 21% of the atmosphere, that's a relative change of only about 1 part in 150 of the oxygen content, which isn't a big deal. Air pressure already varies by more than that (it's the equivalent of about 7 mb of pressure, roughly the difference between a mild storm and a clear day) as weather systems pass by, so your body is already well-adapted to handling such small changes in oxygen content.

(Actually, I wonder if typical sea-level pressure is a bit higher today than it used to be. CO2 is heavier than oxygen, so the atmosphere should "weigh" slightly more than it used to - by a factor of, give or take, about 0.07 ppt. That's not nothing! It should correspond to a global increase in surface atmospheric pressure of about a millibar, which should be detectable. [EDIT: okay, a tenth of a millibar is less.])