Stone has always been a challenge, was just attempting to drill in a blue stone foundation and was very difficult, all I can say is keep drilling and take breaks, have a spray bottle and spray water on the bit as you drill and ensure you have a concrete bit. Brick and soffit are much easier...