Background: Hepatocellular carcinoma (HCC) is the most common primary liver malignancy with high morbidity and mortality worldwide. Tumor immune microenvironment (TIME) plays a pivotal role in the outcome and treatment of HCC. However, the effect of immune cell signatures (ICSs) representing the characteristics of TIME on the prognosis and therapeutic benefit of HCC patients remains to be further studied. Materials and methods: In total, the gene expression profiles of 1,447 HCC patients from several databases, i.e., The Cancer Genome Atlas (TCGA), International Cancer Genome Consortium, and Gene Expression Omnibus, were obtained and applied. Based on a comprehensive collection of marker genes, 182 ICSs were evaluated by single sample gene set enrichment analysis. Then, by performing univariate and multivariate Cox analysis and random forest modeling, four significant signatures were selected to fit an immune cell signature score (ICSscore). Results: In this study, an ICSscore-based prognostic model was constructed to stratify HCC patients into high-risk and low-risk groups in the TCGA-LIHC cohort, which was successfully validated in two independent cohorts. Moreover, the ICSscore values were found to positively correlate with the current American Joint Committee on Cancer staging system, indicating that ICSscore could act as a comparable biomarker for HCC risk stratification. In addition, when setting the four ICSs and ICSscores as features, the classifiers can significantly distinguish treatment-responding and non-responding samples in HCC. Also, in melanoma and breast cancer, the unified ICSscore could verify samples with therapeutic benefits. Conclusion: Overall, we simplified the tedious ICS to develop the ICSscore, which can be applied successfully for prognostic stratification and therapeutic evaluation in HCC. This study provides an insight into the therapeutic predictive efficacy of prognostic ICS, and a novel ICSscore was constructed to allow future expanded application.