I think it depends on a lot of factors. In general, I believe that adipose tissue (fat) retains heat better than a lot of other tissues, so I think it depends somewhat on an organism’s bodily composition. For ectotherms, they do need circulation to distribute warmth around to all their tissues, and a bigger body means that process naturally is going to take somewhat longer. This is part of why crocodiles (which in many cases are really big animals) are all in aquatic sit-and-wait predatory niches in their surviving forms, as they can be huge but still relatively sluggish and spend a lot of time basking, because they just need enough energy to strike out hard when the moment is right, they don’t need to pursue prey, etc. There used to be fully-terrestrial predatory lineages of crocodilians, but they were in general not quite as big and were built very differently, being generally lean and long-legged for pursuing land-based prey (they were probably a lot like big monitor lizards like Komodo dragons in terms of the niche they occupied and how they hunted, and they were probably similarly limited to being diurnal hunters in hot climates).

Like I know that, at least for big predatory dinosaurs, the assumption that they were ectothermic like living reptiles was long a source of confusion, to the point where (prior to the theory that dinosaurs were probably mostly or entirely warm-blooded) a number of paleontologists forwarded the idea that dinosaurs like T. rex were more likely scavengers than active hunters, because it was just too hard to imagine them being able to relentlessly pursue prey with that kind of bulk if they were cold-blooded.

Clearly, it isn’t a complete barrier against getting really big, as plenty of very large presumably cold-blooded creatures have existed over time (like I believe that most Mesozoic marine reptiles like ichthyosaurs and mosasaurs are assumed to have been ectotherms, since they came from older diapsid lineages that hadn’t evolved endothermy—in fact, mosasaurs seem to have literally been gigantic marine lizards, related to monitors—and they were still able to get pretty massive in some cases, although still not quite as big as the largest whales), but I also think it probably indicates something that the very biggest of animals to ever live all seem to have been warm-blooded, and that’s definitely not that endothermy itself is a barrier to great size, quite the opposite in fact.