Computing distances in cosmology is a bit of a pickle since within relativity there are several different measures of distance all of which have a different physical meaning. This is a direct consequence of relativity. Some of the types of distance you could compute are physical distance, comoving distance, angular diameter distance, and luminosity distance as detailed here https://en.wikipedia.org/wiki/Distance_measure .I will move on and assume that you want to compute the "proper distance" since that is probably closest to what you would conventionally call a "distance" based on the phrasing of your question. As shown in the article I posted above the proper distance is related to the comoving distance as follows: d=dh*a , where d is the proper distance, dh is the comoving distance which ignores the expansion of the universe and is constant in time and "a" which is the scale factor which is an increasing function of cosmological time and is given by the first Friedman equation https://en.wikipedia.org/wiki/Friedmann_equations .

Check out this link. https://jila.colorado.edu/~ajsh/courses/astr2010_22/evol.html the first plot gives you the growth of the scale factor as a function of physical time. Of course the evolution depends on the cosmological model you consider hence the different colored plots. The canonical cosmological model that we currently believe to be correct is the purple line in the plot hence you should focus only on that. If you look at the horizontal axis, it is labeled in "billions" which implies that the growth of the scale factor is extremely tiny, hence the distance would barely be altered (see below). If you choose a billion years the scale factor increases by about 10%, hence, in a billion years both points you mentioned above will be approx 10% further away in proper distance.

In case you are semi-educated in calculus, you should be able to recognize the first Friedman equation as being a simple first order differential equation for function a. By solving this equation with initial condition a(t=0)=1 you should be able to find a(t=1 million years). We need a few more elements before we do that. We need to specify the cosmological parameters that enter the 1st Friedman equation. We set k=0 because as far as we know our universe is devoid of curvature (it is flat) and then on the right hand side we assume that the first term scales as 1/a^3 because it represents the dark matter and the second term is constant (dark energy). We also need to ensure that evaluated today, the first term is 1/2 of the second one roughly speaking since there is twice as much dark energy compared to dark matter in the present moment. You will also need the critical (total) energy of the universe from https://pdg.lbl.gov/2020/reviews/rpp2020-rev-astrophysical-constants.pdf . If you put everything together you find the scale factor for the two times you mentioned above.

a(t= 1 million years)= 1.00007

a(t= 1 billion years)= 1.07

This is essentially a full explanation of how to derive the purple line in the plot. Now that we have the scale factor all we have to do is to multiply it by the distances you mentioned

Andromeda after 1 million years will be about 0.007% light years further away and about 7% in 1 billion years. Of course that is ignoring the "peculiar velocities" which is the technical term of what you mentioned above as "local" velocities