“We actually spent a lot of time facing each other in the nets, so that’s quite cool,” Duffy said of the IPL days