The reason your puppy is digging is most likely out of boredom and a lack of respect for you and your things. I would suggest first getting yourself a trainer. Since you have a border collie, I would suggest taking classes from someone who does competition training wether or not your interested in competition. Competition training is far more interesting than 'pet training'. Plus it sounds like your puppy is pretty independent.
If you can't afford a trainer, then I would first suggest teaching your dog to retrieve. Give your puppy an activity. Retrieving is a great way for you and your puppy to build a relationship.
Secondly, you shouldn't allow your puppy to be in the yard unsupervised. The puppy should only be in the backyard when you're there. NEVER LEAVE PUPPY UNATTENDED. When you notice the puppy beginning to dig, say 'leave it', when they stop give him/her a treat and then start playing with your puppy or give puppy a kong-filled with food/peanut butter.If you are going to leave, place your puppy in a crate. Dogs are quite comfortable in a crate for up to 8 hours.
Thirdly, I would place your puppy on a NO FREE LUNCH PROGRAM, meaning make your puppy work for everything. Before you feed your puppy breakfast or dinner, make the puppy 'lie down' and 'watch', once puppy has accomplished this then place the food bowl on the ground. Make your puppy "wait" at the door. Practice "downs", "sits", and "stays" throughout the day. Don't randomly pet your puppy. By randomly petting your puppy, you are giving him/her a reward for nothing. You don't want to be giving free rewards when you don't have the respect of your dog. You need to become the "pack leader" again. The last thing you need to do is probably the most important thing...and that's to start tethering the puppy to you. Get a leash and tie one end to you and the other to the puppy. Have the puppy tied to you when you are in the house. You should do this for 2 weeks. when your making dinner, watching tv, reading, vacuuming, whatever the puppy should be tied to you. I know this sounds very odd...but trust me this method has been more affective then anything else at gaining the respect of your puppy/dog and getting your "pack leader" status back.
Sorry this was so long...I hope it helps. It has helped me at rehabing dogs for years.