The tape might not burn, but it can melt. Also not all electrical tape is equal - a lot of imported cheap stuff might not fully live up to requirements ;-)

Regarding the rest of the setup it sounds pretty much like a standard oil lamp style of setup. As long as the flame isn’t directly against the glass - and if you hold the wick using the metal lid it isn’t- then you should be good to go :-)

You should perhaps post a picture of what you made or describe it a bit more in detail for us to give proper answers. When you say lantern: Is the flame going to be inside the jar, or outside? Regardless I would not use electrical tape on something where there is going to be a flame - the tape might shrink or even catch fire.