Kelly was born in Castlehacket, Tuam, Co Galway and emigrated to the United States, landing in New York City. He led the famed Irish Brigade at the Battle of Gettysburg. At the outset of the, Kelly enlisted in the Union Army and saw action as captain of Company East of the 69th New York Infantry at the First Battle of Bull Run. He briefly was a captain in the 16th United States. Infantry. On 14 September 1861, he was named lieutenant colonel of the 88th New York Infantry and fought in the Irish Brigade’s major battles in 1862.
He commanded the regiment at the Battle of Antietam. While stationed at Harpers Ferry following the Maryland Campaign, he was promoted to colonel on October 20, 1862. He led the regiment in the ill-fated attacks in front of Marye’s Heights in the Battle of Fredericksburg. Kelly was acting commander of the Irish Brigade at the end of 1862.
After the 1863 Battle of Chancellorsville, Kelly was promoted to command the Irish Brigade following the resignation of Brigadier General Thomas Francis Meagher. Kelly led the heavily depleted brigade (fewer than 600 men) in an attack at the Wheatfield at Gettysburg. The brigade lost 198 of 532 troops engaged, around 37%.
Kelly resumed his role as colonel of his regiment as more senior officers returned to the brigade. However, with the death of Colonel Richard Byrnes at the Battle of Cold Harbor in 1864, Kelly again commanded the brigade. At the age of 42, Kelly died during the Siege of St. Petersburg when he was shot though the head while leading the Irish Brigade forward against Confederate earthworks.
His body was recovered and sent back to New York for his funeral. He was buried in First Calvary Cemetery in Woodside, New York.