The best way to prevent an urban planter box from freezing solid is to keep the soil evenly moist but never waterlogged, improve drainage, and insulate both the sides and the top of the container. In winter, the goal is not to keep the box warm all the time; it is to avoid rapid freeze-thaw cycles that can damage roots and crack containers.
First, check drainage. Make sure every drainage hole is open and that no saucer is collecting water. Empty saucers or raise the box slightly so excess water can escape. Use a free-draining potting mix rather than dense garden soil, because soggy soil expands when it freezes and puts more pressure on roots and the planter.
Second, insulate the planter. Wrap the outside with bubble wrap, burlap, old blankets, straw, or rigid foam board, leaving the top open for airflow. For extra protection, line the inside with a layer of insulation before filling it, or place the pot inside a larger container and pack the gap with straw or leaves. Raise the box off cold concrete or paving stones with bricks or pot feet; this reduces heat loss from the bottom.
Third, protect the soil surface. After the first hard freeze, add a thick layer of mulch, straw, or fallen leaves around the plants, but keep it slightly away from stems. If the planter is on a balcony or roof, use a windbreak to reduce drying and wind chill. Group pots together so they shelter one another.
Water carefully. Plants still lose moisture in winter, especially on windy days. Water when the soil is dry a few inches down, preferably on a mild morning, and avoid letting water sit in the saucer. Do not keep the soil constantly wet. Snow can act as natural insulation, so you can leave a moderate layer on top of the planter rather than brushing it all away.
For very cold or exposed sites, move the planter to a sheltered spot, use a cold frame, or choose a larger, frost-resistant container. Check the box during freeze-thaw spells and repair cracks promptly. With good drainage, insulation, and sensible watering, your urban planter box is much less likely to freeze into a solid, damaging block.