Superficial cracks in bricks are not a serious concern, but if the brick is cracked all the way through, it has to be replaced with a new brick. Similarly small cracks in stone siding won't lead to structural damage, but if left untreated, it might lead to more serious issues in the future.
Recently Reviewed Grimsby Brick or Stone Siding: Repair or Partially Replace