His first books dealt with scientific subjects, and include Physiological Æsthetics (1877) and Flowers and Their Pedigrees (1886). He was first influenced by associationist psychology as expounded by Alexander Bain and by Herbert Spencer, the latter often considered[by whom?] the most important individual in the transition from associationist psychology to Darwinian functionalism. In Allen’s many articles on flowers and on perception in insects, Darwinian arguments replaced the old Spencerian terms. On a personal level, a long friendship that started when Allen met Spencer on his return from Jamaica grew uneasy over the years. Allen wrote a critical and revealing biographical article on Spencer that was published after Spencer’s death.

After assisting Sir W. W. Hunter in his Gazeteer of India in the early 1880s, Allen turned his attention to fiction, and between 1884 and 1899 produced about 30 novels. In 1895, his scandalous book titled The Woman Who Did, promulgating certain startling views on marriage and kindred questions, became a bestseller. The book told the story of an independent woman who has a child out of wedlock.

In his career, Allen wrote two novels under female pseudonyms. One of these, the short novel The Type-writer Girl, he wrote under the name Olive Pratt Rayner.

Another work, The Evolution of the Idea of God (1897), propounding a theory of religion on heterodox lines, has the disadvantage of endeavoring to explain everything by one theory.

Allen also became a pioneer in science fiction, with the 1895 novel The British Barbarians. This book, published about the same time as H. G. Wells’s The Time Machine, also described time travel, although the plot is quite different. Allen’s short story The Thames Valley Catastrophe (published 1901 in The Strand Magazine) describes the destruction of London by a sudden and massive volcanic eruption.

Many histories of detective fiction mention Allen as an innovator. His gentleman rogue, the illustrious Colonel Clay, is seen as a forerunner to later characters. In fact, Allen’s character bears strong resemblance to Maurice Leblanc’s French works about Arsène Lupin, published many years later; and both Miss Cayley’s Adventures and Hilda Wade feature early female detectives.