Investigating an optimal set of features for a machine learning bot playing Schnapsen and how these features influence performance and training time
What follows is the abstract of the research paper which you can download at the bottom of the page.
This paper investigates how different features implemented in a machine learning bot using neural networks impacts the performance and training time of said bot. This is achieved the in the card playing game schnapsen. For this experiment, a framework was provided with which to test the bots in playing schnapsen.
20 different features were implemented in a bot and then ranked according to the RFE method using a linear regression algorithm. In accordance with this, 4 bots were created to test the hypothesis that the 11 best ranked features implemented in a bot will outperform any other bot and achieve at least a 50% win rate against the control bot. The bots are; allFeatures bot which has all the features that were ranked with RFE implement, 11MostRelevant bot which has the 11 highest ranked(rank closest to 1) features, Mixed bot which contains the six highest ranked features as well as the five features ranked directly below the eleven highest ranked features, and the 11LeastRelevant bot which has the 11 lowest ranked(ranks farthest away from 1) features. The hypothesis that “the training time of a bot is affected both by the total amount of features in the feature set as well as the existence of irrelevant features” was also tested.
To determine which bot performed best they played 1000 games against the control bot each. In contrast to the aforementioned hypothesis, the 11MostRelevant bot did not perform the best. The allFeatures bot had the highest number of wins and the 11LeastRelevant had the least amount of wins as was expected. The lowest training time was achieved by the Mixed bot, which was 4 times faster than the second fastest 11MostRelevant bot. This may be because when certain features are hot-encoded, they are represented in a binary vector, which can have many values in it. The Mixed bot does not have a feature, which requires that much space and hence was faster.