In this work, we use Monte Carlo simulations to examine the depletion of two-dimensional hard squares and rods by hard disks. We determine the effect of depletant size and concentration, as well as the aspect ratio (squares vs. rectangles) of the large particles on the potential of mean force. We find that the disks are able to deplete squares and rectangles of certain aspect ratios. The strength of the attraction increases with increasing depletant concentration and depends on depletant size. By subtracting the potential of mean force for two large objects in the absence of depletant from that in the presence of depletant, we are able to separate the effect of rotational entropy from that of depletant. We also probe the relative orientation that two squares adopt as they approach each other in a sea of depletant. We observe rich behavior, in which this relative orientation varies with inter-particle separation (e.g., for some separations, particles adopt a face-to-face orientation, while for others, the vertex-vertex orientation is preferred) and depends on depletant size and concentration. This work contributes to the understanding necessary to achieve self-assembled structures that possess complex, long-range order from simple building blocks.