Impact of Timing in the Design Process on Students’ Application of Design for Additive Manufacturing Heuristics

As manufacturing methods evolve, so do the challenges associated with designing components for these manufacturing methods. Additive manufacturing (AM) is one of these modern manufacturing methods that have a unique set of rules for designers. In particular, it can be difficult for novice or student designers to succeed at design for additive manufacturing (DfAM), due to the unintuitive nature of some DfAM design techniques. Even designers who are experts in designing for traditional manufacturing do not have an advantage over novice designers when designing for additive manufacturing [1], due to a lack of formal education or training as a result of the recent surge in popularity of the technology, although training curricula are being developed and tested for both novices and experts [2,3]. In order to gather information to assist designers, sets of heuristics or rules of thumb for designers have been collected from expert knowledge, award-winning products, and databases of 3D printed parts. In traditional design contexts, being exposed to design heuristics helps novices create designs of higher novelty and quality, indicating that heuristics are a valuable training tool for the general design field. Multiple works by other authors [4–6] have identified DfAM-specific heuristics; however, little work has been done to test or validate these heuristics as educational tools.

This paper quantitatively studies the way student designers use DfAM heuristics. In addition to testing the validity of heuristics as a training tool, it will study the effects on the way novices apply DfAM heuristics when they are received at different stages in the design process. Previous work on creative problem-solving has revealed that solutions to problems differ when helpful information is presented at different points in the problem-solving process [7], but little work has been done to relate this phenomenon to heuristics, especially DfAM-specific heuristics. It is important to study heuristics in this context because 3D printing has become a critical asset to the rapid prototyping phase of the design process; depending on the design problem, a 3D printed part may be the end product resulting from the final stage of the process. This paper will enhance the understanding of the relationship between heuristics and the design process, allowing for more effective workforce training and education. The focus of this paper is based on the following research question:

The question deals with whether it is best to present heuristics before or after an open goal has been established. Background knowledge in 3D printing may lead to solutions that are more appropriate for additive manufacturing as well as ideas that are more creative [8]. The design process is a cyclical, evolutionary process, and it is desirable to understand what factors affect that evolution, for the purpose of training novices as well as performing design work. In answering this question, the capabilities of DfAM heuristics as training and educational tools will be examined. This will contribute to the development of DfAM-specific curriculum, with the benefits of compiling a more standardized set of topics as well as determining appropriate timing for training.

Additive manufacturing is a manufacturing process that produces parts by depositing material layer-by-layer. AM is a decades-old technology that has experienced a recent surge in popularity due to innovations increasing the affordability and accessibility of 3D printing technology [1]. Often, the terms “additive manufacturing” and “3D printing” are used interchangeably. This paper will deal specifically with a type of 3D printing called fused filament fabrication (FFF). In FFF, a thin filament of plastic material is fed into a heated print head, which melts the filament and extrudes it into layers [9]. This form of AM is the most popular technology on the market because both the machine and materials are highly economical, and no chemical post-processing is required [10].

Design for X is a technique for new product development. It is the practice of proactively coordinating and communicating requirements, design goals, and constraints for a project. There are many possible “X-factors”—manufacturing, assembly, quality, logistics, service, environment, sustainability, etc. Concurrent engineering is the practice of considering all relevant X-factors at the beginning of the design phase [11]. Of the possible X-factors, manufacturing and assembly are critical to this work. Design for Manufacturing (DFM) saves costs and time by integrating manufacturing considerations early into the design process. These manufacturing considerations are often combined with assembly considerations in Design for Manufacturing and Assembly (DFMA), which focuses on simplifying products, reducing assembly and manufacturing costs, and quality improvement. DFMA may consider the fastening method, part size, assembly instructions, design modularity, and use of standard components [12]. The application of the additive manufacturing X-factor to DFMA principles results in a new process—DfAM. The purpose of DfAM is defined as the “synthesis of shapes, sizes, geometric mesostructures, material compositions, and microstructures to best utilize manufacturing process capabilities to achieve desired performance and other life-cycle objectives” [13].

Designers utilizing DfAM face the challenge of incorporating the unique considerations required to create additively manufactured parts compared with the familiar process of designing for traditional manufacturing methods [1,4]. For example, part complexity and internal geometry is a critical consideration when designing for mold-based production techniques, but with AM, the complexity of a part has little effect on its cost or manufacturability [14]. This allowance also permits the use of complex internal geometry and new technologies, such as topology optimization [15]. Other benefits of AM include diverse material choice—usable materials include polymers, metals, ceramics, sand, clay, concrete, custom alloys or composites, and even food or biological materials, such as cells and proteins. In addition to customizing the material, additively manufactured products can be custom-fit, personalized, or customized, which is particularly useful in the medical device field [15]. Designers who make good use of AM technologies can remove the need for post-manufacturing assembly by consolidating parts, embedding objects, and directly producing assemblies and interlinked parts [15]. Dinar and Rosen have proposed a list of criteria that can be applied to existing design features to improve them using AM technologies, including part consolidation, weight reduction, structural strength, geometric complexity, service life, and production volume [16].

Conversely, there are drawbacks to AM that are not present in traditional manufacturing methods. In order to create an additively manufactured part, a comprehensive digital model is needed [15]. This model is typically a fully constrained computer-aided design (CAD) model, which limits part complexity based on the abilities of the CAD program or skills of the designer [15]. Parts created through AM are impacted by anisotropy—the part’s properties vary based on how it was oriented on the build plate. Build plate orientation impacts the build time, dimensional accuracy, cost, surface finish, and support structure of the part [17]. Support structures may be needed during part production to ensure the part is able to withstand mechanical and thermal stresses during the build process [15]. Support structures increase the amount of material used, time required for part manufacturing, and the time and complexity of post-manufacturing operations [18].

Design heuristics can be used as an assistive tool for designers to avoid common pitfalls of AM. According to Fu et al. [19], a heuristic is a “context-dependent directive, based on intuition, tacit knowledge, or experiential understanding, which provides design process direction to increase the chance of reaching a satisfactory but not necessarily optimal solution.” The influence of design heuristics as a concept generation tool has been heavily studied. Yilmaz et al. [20] developed 77 design heuristics by examining award-winning products and patents [21], concepts generated by an expert designer [22], and think-aloud protocols from designers working on a product design task [23,24]. These heuristics were tested by presenting them to novice designers as cards and analyzing the designs they created. Results indicated that exposure to design heuristics may result the development of higher-quality, more creative designs.

In the concept generation phase, heuristics appeared to stimulate divergent thinking, resulting in more diverse and creative designs generated by novice designers [25]. Concepts generated without design heuristics were less developed and tended to be less original, often consisting of replications of existing products or ideas with minor changes [26,27]. A comparison of ideas that were generated using design heuristics against brainstormed ideas developed using no ideation tools indicated that designers using design heuristics considered a different problem scope than those without. The brainstormed concepts tended to focus on holistic systems, while concepts generated with design heuristics focused on component design and how to achieve specific functions [28]. Further studies followed subjects through the design process and found that the influence of design heuristics could be detected not only in the idea generation phase but also during the prototyping and iteration phases [29].

Recently, efforts have been made to expand the field of heuristics into the field of design for additive manufacturing, with the intention of using heuristics as a method of quickly transferring AM knowledge to designers. Compared with general design heuristics, DfAM heuristics are more reliant upon the material, AM process, and specific machine being used [4]. Adam and Zimmer [5] developed a restrictive AM design rule catalog based upon experimentation to determine parameters and guidelines for part features by evaluating the success of manufacturing various features. Urbanic and Hedrick [6] compiled a list of general rules specifically for FFF processes, while Blösch-Paidosh et al. [4] analyzed databases of existing designs for 3D printing to derive 29 AM heuristics based on key functions and features.

A user study was performed by Blösch-Paidosh and Shea [30] around these 29 opportunistic heuristics in which novice designers were presented with a redesign task. Results showed that participants who had access to heuristic cards used more heuristics in their redesign and made more modifications to the initial design, although the quality and feasibility of these designs was not evaluated. An additional study confirmed that participants in possession of opportunistic DfAM heuristic cards made more AM design modifications, and additionally found that redesigns developed with the help of heuristics were more novel than those developed without [8].

Perez et al. [31] used a similar heuristic extraction method to develop a set of 23 design principles from the features of the “most popular” designs from Thingiverse.com, a crowd-sourced repository of digital design data. Heuristics were a combination of opportunistic heuristics and nonprocess-dependent restrictive heuristics. Participants in a study were given access to four of these heuristic cards, and subsequent designs increased in quality and novelty compared with initial designs created without heuristic information. However, the effect of heuristic access on the variety of solutions developed, as well as the performance of a control group, was not examined in this study [32].

Prabhu [33] presented DfAM principles in a lecture format, with distinctions made between opportunistic and restrictive principles. While neither form of heuristic affected the creativity of design outcomes, lectures featuring both opportunistic and restrictive DfAM principles were found to result in ideas with higher technical goodness compared with ideas generated by participants who had not been exposed to DfAM principles or had only been exposed to restrictive DfAM principles. Results also showed that after receiving DfAM principles, participants simplified their designs in order to comply with the limitations of AM processes [34]. At various universities, pilot courses utilizing lectures on design for additive manufacturing principles have proven successful at improving student engagement and transferring DfAM knowledge to students [35–38].

Given the benefits of heuristic use for design, one of the next steps is to study the way heuristic presentation impacts designers. Presentation modality has been studied to investigate different methods of presenting information to designers, with the findings that presentation modality does not affect the quality and novelty of designs, although AM rules presented as text were perceived as more difficult to understand [1] and less appreciated than physical artifacts and images. Designers found AM knowledge more useful when it was presented as an opportunity afforded by AM rather than a restriction of AM [39].

Fillingim et al. [1] compared the performance of design experts and novices in a redesign activity using restrictive heuristics and found that while experts generated higher-novelty redesigns, there was no difference in quality between the groups. Laverne et al. [40] found similar results—concepts generated by experts were more original, but less manufacturable. In the same study, novices proposed more concepts than experts. Yang et al. [41] determined that novices presented with AM knowledge generated a higher quantity, quality, and novelty of architectural innovations, but the AM knowledge did not have a significant effect on radical innovations, indicating that it may be beneficial to present AM information as a function-behavior-structure model to provoke radical design. A factor that remains to be studied in the context of DfAM heuristics is timing.

The timing of information presentation impacts the way in which information is used by designers [7]. Information related to a problem impacts idea generation more if it is presented when there is already a problem statement, or “open goal.” In word association problems, subjects who had previously failed to solve a problem and then were given a hint were more likely to successfully solve the problem than subjects who were presented with the hint before attempting the problem for the first time [42]. The hint was more effective after the open goal had been established. Tseng et al. [7] studied the impact of open goals on design problems and found that when participants were presented with information distantly related to a design problem after working on the problem for five minutes, solutions produced were greater in quantity and in novelty compared with solutions produced by participants who were presented with the distantly related information before being given the design problem. Therefore, the presence of an open goal increases the likelihood that a designer will incorporate a given piece of information into their problem-solving.

Participants that are primed with information that is similar to a design problem, such as descriptions of devices that perform similar functions to the function required by the design problem, generate more novel ideas compared with participants who do not receive priming information; although, solutions within the primed group are more functionally similar to each other [7]. Even information that is only distantly related to a problem can prime solution concepts when presented during a break in problem-solving; although the priming effect is stronger when information is more closely related to the problem being solved [7]. Priming is heavily related to design fixation, defined as “a blind adherence to a set of ideas or concepts limiting the output of conceptual design” [43]. Evidence of design fixation has been identified by examining solutions for similarity to examples provided alongside a design problem [43,44].

Another timing effect related to problem-solving is the incubation effect, which is particularly strong in creative divergent thinking tasks with no correct answer. The incubation effect causes designers to gain creative insight on a problem, occasionally even “spontaneously” coming up with a solution, during a break in problem-solving [45]. In many domains, the unconscious thinking that occurs during incubation is considered superior to conscious thought when the length of the incubation period is appropriate [46], with more creative ideas developed as a result of an incubation period [47]. A study of incubation effects on design problems found that solutions developed after an incubation period were higher in novelty and in quantity, but the incubation period did not aid the designers in coming up with more feasible solutions [48]. The increase in novelty and variety indicates that the incubation period helped to break the designers’ fixation to their initial designs, which were influenced by examples.

The participants recruited for this study were undergraduate students from four class sections of an introductory mechanical engineering design course at the Georgia Institute of Technology in Atlanta, Georgia, USA. In a 90 min intervention during their regular virtual class period, all students from the four sections received a lecture covering DfAM heuristics and participated in two design activities. Students were given an electronic consent form and offered extra credit in the class as compensation for taking part in the study. Those who did not agree to sign the consent forms still participated in all activities, but their data was removed from the set of data that was analyzed. Those who wished to earn extra credit but did not wish to participate were given the opportunity to complete an alternative assignment for extra credit.

After introducing the study and obtaining consent, the researcher used a script to navigate participants through the remainder of the study. First, all students were given ten minutes to take an online pre-assessment. The pre-assessment contained self-efficacy questions relating to DfAM as well as quiz-type questions covering key DfAM topics, such as filling in blank heuristics or determining print orientation for a certain part. Self-efficacy questions were developed via pilot studies and expert review and were based off information conveyed in the heuristics lecture and gauged participants’ comfort levels in designing a part specifically for 3D printing, determining if a part is a good candidate for 3D printing, understanding the different types of 3D printing, determining the orientation of a 3D printed part, determining when support structure should be added to a part, and choosing part infill. Participants were asked to rate their level of comfort in performing these AM-related tasks on a 1–5 Likert scale, with 1 representing “Extremely Uncomfortable” and 5 representing “Extremely Comfortable.”

After these first steps, the procedure varied slightly for the different experimental groups. Because the goal of the study was to identify the way that timing of heuristics impacts the designs of the participants, two points in the design process were identified during which heuristics could be administered: (1) before the design problem is given, and (2) after the initial design phase, but before the iterative redesign phase begins. The group that received the heuristics lesson at the beginning of the experiment, before a design problem had been established, was the Heuristics-First group. The Heuristics-First group received the DfAM heuristics lecture after the pre-assessment, followed by a 10 min design problem activity. After the first design activity, students in this group were given a 5 min break before beginning a 10 min redesign activity, which was an opportunity to revisit and redesign the solution generated during the first design activity. The Heuristics-Between group went from the pre-assessment to the first design activity, then received the DfAM heuristics lecture during the break, then completed the redesign activity. Literature indicates that giving designers a break in the middle of a design problem can lead to increases in novelty [48]. In order to account for this, a third group was added as a control. The group was not exposed to the heuristics until after both design phases, in order to get an accurate measure of how novelty and quality changed between the design and redesign phases. Figure 1 shows a visual timeline of the three conditions.

After completing these tasks, all of the experimental groups were given 10 min to take the post-assessment, which was identical to the pre-assessment. Finally, they were given the homework assignment to create a CAD model of their redesign activity solution in Solidworks and upload it into Cura, a 3D printing software, as homework due within 7 days. Participants were instructed to adhere to their sketch as much as possible when modeling the design. Cura allows the user to import a 3D model onto a virtual print bed and view a time estimate, support material, and other features of the print [49]. The students submitted the 3D object file generated by Cura and a screenshot of the simulated print bed showing the estimated print time, filament usage, and amount of support material.

DfAM heuristics and rules used in the presentation were selected from various larger sets of DfAM rules [1,4], as well as existing classroom curricular materials [50]. In total, the presentation consisted of 11 slides of restrictive heuristics and two slides of opportunistic heuristics. A written list of the heuristics may be found in the Appendix. The lesson first gave a brief introduction to additive manufacturing, then went into the specifics of 3D printing and when 3D printing is appropriate to use. The main part of the presentation covered one AM heuristic or rule per slide, making use of written explanations as well as figures and photos, similar to the format of a heuristic card, to clarify the heuristics as shown in Fig. 2.

The objective of the design problem was to design a 3D-printable soap dish, based on the study by Fillingim et al. in which participants redesigned a 3D printed soap dish [1]. The task was accompanied by the specifications of the printer on which the soap dish would be printed, including the size of the print bed and layer resolution of the printer. The precise wording of the task was

The redesign problem was stated as follows:

The soap dish design problem was chosen because it is simple enough for beginner students to sketch and model in CAD within the allotted timeframe, yet there are certain complexities making it appropriate as a DfAM challenge. Fillingim et al. [1] chose a soap dish redesign problem due to its compatibility with support structure-related heuristics.

In total, 68 students signed the consent form and agreed to participate in the study. The Heuristics-First (20 participants) and Control (17 participants) groups were made up of one class section each, while the Heuristics-Between group (31 participants) was made up of two smaller class sections combined. Participants were given the option to fill out a demographic survey before or after the study. The study consisted of 68 student participants, although four of them did not fill out the demographic survey. Of the 64 students who completed the survey, 16 were women, 47 were men, and one did not provide their gender. There were 44 participants aged 18–20, 16 aged 21–23, 3 aged 24–26, and one participant over 30 years old. While 1 participant chose not to provide their race, 11 identified as Asian, Native Hawaiian, or Other Pacific Islander, 37 identified as white, six identified as Black or African-American, four identified as Hispanic or Latino, and five identified as multiracial. Although two students were pursuing a business major, 62 of them were pursuing a degree in mechanical engineering. The participants were in varying stages of degree completion: 24 were in their second year, 29 in their third year, 10 in their fourth year, and one participant was in their fifth year of undergraduate studies.

Two researchers refined and utilized coding schemes for evaluating the quality and novelty of the design sketches developed during the study period, based on the metrics developed by Fillingim et al. [1]. For quality, five criteria were used to judge a design’s suitability for 3D printing, as well as its ability to carry out its intended function. Designs were assigned a positive score of +1, a neutral score of 0, or a negative score of −1 for each criterion. The criteria are described in detail below.

The soap dish design was required to fulfill two functions: (1) allow water to drain away from the bar of soap, and (2) secure the bar of soap so that it does not slip away. A positive score was given to designs that fulfilled both functions, a neutral score was given to designs that fulfilled either one of the functions but not both, and a negative score was given to designs that failed to fulfill both of the functions.

The designs must not only be able to withstand the printing process but also the everyday use of being in the shower. Two requirements of print strength were identified: (1) all components are of sufficient thickness (based on the specifications of the printer) and well-attached to the main design and (2) there exists a print orientation which will allow the part to print without failure. As with the Functionality criterion, a positive score was given to designs that fulfilled both requirements, a neutral score was given to designs that fulfilled either one of the requirements but not both, and a negative score was given to designs that failed to fulfill both of the requirements.

It was determined that a design is of higher quality if it requires less support material during 3D printing. This reduces the total amount of material used, as well as time spent afterward removing the support material. Designs requiring no support material received a positive score. Designs for which support material was needed, but adjustments to print orientation could reduce or eliminate the need for support material were given a neutral score. A negative score was given to designs that required excessive amounts of support material that may be difficult or impossible to remove.

The goal of the design task was to design a soap dish to be additively manufactured, so it is ideal for the design to be able to function as intended without the need for any additional off-the-shelf (OTS) parts. A positive score was given to designs that required no additional components besides those that were 3D printed. Designs were given a neutral score if the design required the purchase and installation of one non-AMed item, and a negative score if the design required the purchase and installation of more than one non-AMed item. One non-AMed item was set as the threshold for the neutral score upon inspection of the design sketches revealing that due to the inherent simplicity of the design problem, the majority of the designs used zero or only one non-AMed component.

A design that requires little assembly is considered of higher quality due to its ease of use and reduced failure points. When considering the number of assembly steps for a design, the assembly of both 3D printed parts and off-the-shelf components was considered. A positive score was given to designs that require no assembly before use. A neutral score was given to designs requiring one assembly operation, while a negative score was given to designs requiring more than one assembly operation. An assembly operation was defined as the number of different steps that needed to be taken to fully assemble the design. For example, assembling two 3D printed parts and adding a suction cup attachment would count as two assembly operations, whereas adding two suction cups would only count as one distinct assembly operation. Again, these thresholds were chosen based off of the specific design problem.

Two researchers independently examined and rated the quality and novelty of 25% of the data. Cohen’s Kappa was used to evaluate inter-rater agreement to determine if the quality rubric had an acceptable level of detail and repeatability. Inter-rater agreement across the quality criteria resulted in 88.8% agreement and a sufficient Cohen’s Kappa of 0.74, so one researcher coded the remainder of the data for quality.

The functionality of the soap dish was considered the most important aspect of its quality, because the designs must be able to perform the tasks that they are assigned to. As a result, functionality was given a weight of 50% toward the total quality score, while the other four categories were weighted 12.5% each. The resulting quality scores were then normalized to range between 0 and 1.

One of the two required functions of the soap dish was to allow water to drain away from the bar of soap. The three main drainage solutions that emerged were drainage holes going through the dish, a sloped surface, and openings along the edges of the dish. Any combination of these solutions was also possible. In addition, the through holes were further categorized as either holes or slots, and as elevated or not elevated. To be considered elevated, the holes must allow water to drain into an unenclosed area; if the holes would fill with water during use and cause the soap to remain wet, the holes were considered not elevated.

The second required function of the soap dish was to hold the bar of soap to prevent it from slipping off the dish. While some participants chose not to address this function at all, most chose to contain the soap within a curved surface or walls that surrounded the bar of soap on all or most sides.

Some participants chose to separate their design into multiple parts for ease of 3D printing or to manufacture parts that would otherwise have to be purchased OTS. This category was based off of the number of 3D printed parts comprising each design, ranging from one to six. Novelty scores were assigned according to the number of other designs that used the same number of AMed parts.

The mounting style refers to the way the soap dish interacts with the shower or bathtub. Participants often designed their product to sit flat on the edge of a tub or other flat surface. Others chose to hang it or mount it to the wall in some fashion, while a few participants chose to make their design convert between a wall-mounted and flat-laying configuration.

Designs required a wide variety of off-the-shelf components to interface with the soap dish. These components included suction cups, adhesive, screws, rubber feet, magnets, hooks, and other unspecified mounting hardware.

Again, 25% of the data was evaluated by the two researchers. This resulted in 93.75% agreement, so one researcher coded the remainder of the data for novelty. For each design, the novelty scores from each of the five categories were averaged together to obtain a total novelty score. Each category was weighted equally, since none of them were deemed more important than any others.

With regard to quality, the Heuristics-Between group is expected to score the highest, then the Heuristics-First group and then the Control group, respectively (Hypothesis 1A). The Control group is expected to score low because they will not have received the heuristics lesson and therefore may find it challenging to design parts that are well-suited for additive manufacturing. Their designs are not expected to increase in quality between the design and redesign phases of the study (Hypothesis 1B). A previous study shows that information related to a design problem is more helpful when a goal or design problem has already been established [7], so it is expected that the heuristics lesson will impact the Heuristics-Between group’s designs more than the Heuristics-First group’s designs, causing the quality of the Heuristics-Between group’s designs to increase significantly (Hypothesis 1C). Because both design phases take place after the heuristics presentation, the quality scores of the Heuristics-First group are not expected to increase (Hypothesis 1D).

It is expected that the Heuristics-Between group will make more design changes during the redesign phase than the Heuristics-First group or the Control group (Hypothesis 2), due to receiving the heuristics lesson between the design sessions. It is believed that at the end of the redesign phase, the Control group will have the highest novelty scores, with the Heuristics-Between group next and the Heuristics-First group having the lowest scores (Hypothesis 3A). It is believed that participants exposed to heuristics in the design phase of the study (the Heuristics-First group) will have the lowest novelty scores due to the priming effect of the heuristics lesson [7], although designs are expected to become more novel after the break (Hypothesis 3B) [48]. Participants in the Heuristics-Between group are expected to produce higher-novelty designs in the design phase that become less novel after the heuristics presentation (Hypothesis 3C), due to the tendency of designers to simplify their designs after exposure to DfAM heuristics [33]. The novelty scores of the Control group are expected to be similar to those of the Heuristics-Between group in the first design phase, and these scores are expected to increase after the break during the redesign phase (Hypothesis 3D).

SPSS Statistics software Version 26 was used for statistical analysis. Different statistical tests were needed to test the within-subjects and between-subjects effects on the dependent variables. Testing within-subjects effects implies that related samples are being tested—in this case, tests are being conducted on the effect of time within one experimental group. Between-subjects effects are used with independent samples; the independent samples refer to the three experimental groups in this case. These effects were analyzed for all dependent variables: the pre- and post-assessment results, the quality and novelty of designs, and the Cura 3D printing data.

The six self-efficacy tasks that were most directly related to AM were selected for analysis, revealing that undergoing the DfAM intervention increased students’ perceptions of their AM abilities. All three experimental groups experienced a statistically significant increase in self-efficacy scores between the pre-assessment and the post-assessment, according to the Wilcoxon signed-rank test [52]. The mean self-efficacy score increased from 3.408 to 4.058 (z = 3.296, p = 0.001) in the Heuristics-First group, from 3.382 to 4.108 (z = 3.885, p < 0.0005) in the Heuristics-Between group, and from 3.206 to 4.255 (z = 3.412, p = 0.001) in the Control group, as shown in Fig. 3.

The Kruskal-Wallis H test [52] was run to determine if there was a significant difference in the Likert score increases between the three experimental groups. Between the pre- and post-assessment, there was a mean increase of 0.650, 0.726, and 1.049 for the Heuristics-First, Heuristics-Between, and Control groups, respectively. There was no significant difference between the groups’ scores in the pre-assessment (χ²(2) = 0.283, p = 0.868), post-assessment (χ²(2) = 2.932, p = 0.628), or amount by which the scores increased (χ²(2) = 2.042, p = 0.360).

The pre- and post-assessments also contained questions to test the participants’ knowledge of AM concepts. Question #8 covered print orientation in the context of maximizing the chance of a successful print. According to McNemar’s test [52], there was no significant increase in the proportion of participants in the Heuristics-First group (z = 2.250, p = 0.125) or the Control group (z = 0.000, p = 1.000) who answered the question correctly between the pre- and post-assessment. In the Heuristics-Between group, there was a significant increase in the proportion of correct answers from 42% in the pre-assessment to 61% in the post-assessment (z = 4.167, p = 0.031). The test of two proportions [52] revealed that there was no significant difference in the scores of the groups at either the pre-assessment (χ²(2) = 4.434, p = 0.109) or post-assessment stage (χ²(2) = 0.536, p = 0.765).

Question #14 of the assessment asked students to fill in a heuristic on the topic of surface finish. The test of two proportions revealed that there was no significant difference in the scores of the groups at either the pre-assessment (χ²(2) = 0.291, p = 0.865) or post-assessment stage, (χ²(2) = 1.043, p = 0.594). There was no significant increase in the proportion of participants in the Heuristics-First group who answered the question correctly between the pre- and post-assessment, (z = 0.900, p = 0.344). In the Heuristics-Between group, there was a significant increase from 48% to 81% in the proportion of correct answers (z = 8.100, p = 0.002). The Control group also had a significant increase in correct answers, from 47% to 88% (z = 5.143, p = 0.016).

The quality scores of each experimental group at the design and redesign phases are shown in Fig. 4. Of the 20 participants in the heuristics-first group, four participants increased their quality score from the design to redesign phase, while one participant decreased their quality score. The increase in mean quality score from 0.791 to 0.819 was not statistically significant (z = 1.084, p = 0.279). Of the 29 participants in the heuristics-between group, ten participants increased their quality score from the design to redesign phase, while five participants decreased their quality score. The increase in mean quality score from 0.765 to 0.774 was not statistically significant (z = 0.400, p = 0.689). Of the 17 participants in the control group, four participants increased their quality score from the design to redesign phase, while two participants decreased their quality score. The increase in mean quality score from 5.302 to 5.474 was not statistically significant (z = 1.382, p = 0.167).

Mean quality scores were not statistically significant between groups in the design phase of the experiment (χ²(2) = 1.122, p = 0.571) or in the redesign phase of the experiment (χ²(2) = 1.634, p = 0.442). Additionally, the amount by which quality scores increased within each experimental group was not significantly different between groups (χ²(2) = 0.009, p = 0.996).

Because the quality metric was based off of five categories, it was also important to look at the categories separately. “Print Strength” and “Support Material” were the two categories that were directly related to the success and quality of a 3D print (referred to hereafter as “Printable Quality”). Therefore, these quality scores were isolated and averaged together, then normalized on the 0 to 1 scale, shown in Fig. 5. The Heuristics-First group did not experience a significant change in Printable Quality (z = −0.587, p = 0.557); neither did the Control group (z = −0.486, p = 0.627). However, between the design and redesign phase, the average Printable Quality score of the Heuristics-Between group significantly increased from 0.741 to 0.862 (z = 2.177, p = 0.029).

Mean Printable Quality scores were not statistically significant between groups in the design phase of the experiment (χ²(2) = 0.301, p = 0.860) or in the redesign phase of the experiment (χ²(2) = 2.780, p = 0.249). Additionally, the amount by which Printable Quality scores increased within each experimental group was not significantly different between groups (χ²(2) = 4.913, p = 0.086).

The following figures are an example of the series of designs submitted by one participant. In the design phase, the quality score of the design shown in Fig. 6 was 0.688, below the average of the group, due to its angled sides that would require support material to manufacture, as well as its failure to allow water to drain away from the soap dish. The redesign in Fig. 7 shows that the subject improved the quality of the design to a score of 0.875 by elevating the bar of soap above the lower surface by the addition of feet; although this operation required the use of increased support material, despite the reduction of the vertical angle of the sides. Figure 8 shows that with only minor changes, this design has been modeled in CAD and prepared for 3D printing using Cura.

In the Heuristics-Between group, there was a statistically significant decrease in the mean novelty score from 0.482 in the design phase to 0.457 in the redesign phase (z = −2.109, p = 0.035), as shown in Fig. 9. There was no statistically significant change in mean novelty scores for the Heuristics-First group (z = −1.853, p = 0.064) or the Control group (z = −0.450, p = 0.653).

Mean novelty scores were not statistically significant between groups in the design phase of the experiment (χ²(2) = 2.759, p = 0.252) or in the redesign phase of the experiment (χ²(2) = 4.865, p = 0.088). Additionally, the amount by which novelty scores changed within each experimental group was not significantly different between groups (χ²(2) = 0.624, p = 0.732).

While coding data for novelty, it was observed that some participants had chosen not to make any changes to their soap dish design during the redesign session. Five out of 29 participants in the Heuristics-Between group chose to keep their design as it was, compared with eight out of 20 participants in the Heuristics-First group and five out of 17 participants in the Control group.

With the data that were coded for novelty, the variety within each experimental group could also be calculated. Variety indicates how different from one another the solutions within a group are, rather than how unique a solution is compared with all the other solutions generated [51]. The total number of unique solutions identified during novelty coding was summed. Variety was calculated for each experimental group as the proportion of the total number of solutions that occurred within that group, as shown in Fig. 10. For example, in the design session, there were 35 unique solutions identified between the six novelty categories. Narrowing the focus to just the Heuristics-First group, it was found that 22 out of those 35 solutions were found among the Heuristics-First group’s pool of solutions. Therefore, the variety score for the Heuristics-First group’s design session was 22 divided by 35, or 0.629. A higher variety score indicates that a higher number of unique solutions occurred within that group, while a lower variety score indicates that fewer unique solutions were developed in that group.

The three pieces of data collected from the participants’ 3D printing follow-up assignment were (1) the amount of filament used to print the soap dish, (2) the amount of time needed to complete the print job, and (3) the percentage of the total print time dedicated to printing support material. Because these data were collected once, at the end of the experiment, there was only one data point per participant, so only the differences between the three experimental groups could be analyzed. For analysis purposes, outliers were removed from the data set. Outliers were defined as data points that were over 1.5 times the interquartile range away from the median and were identified by examination of a box-and-whisker plot. This was done because it was difficult to determine if extreme data points were genuine outliers or due to errors by the participants in modeling and using the printing software, as some of them had little to no experience in doing so.

Analysis indicated that there was a statistically significant difference between groups in the amount of support material (χ²(2) = 7.782, p = 0.020) used, as shown in Fig. 11. Pairwise analysis revealed that the amount of support material used by the Heuristics-First group (mean = 7.4%) was significantly higher than the Heuristics-Between group (mean = 1.9%, z = 13.477, p = 0.016). However, there was no significant difference between the groups in the print time (χ²(2) = 3.574, p = 0.167) or amount of filament used (χ²(2) = 5.013, p = 0.082).

The above results describe the main findings from the pre- and post-assessments. The self-efficacy portion revealed that in all three groups, the participants felt more comfortable in their abilities to perform DfAM-related tasks after the intervention. This was expected, as all three groups were exposed to the DfAM heuristics lesson between taking the pre-assessment and post-assessment.

This finding was reinforced by an improvement in participants’ performance on the quiz-type questions in the assessments. Of two assessment questions testing objective knowledge of DfAM heuristics, one had a significant increase in the proportion of participants in the Heuristics-Between group who answered it correctly, while the second had a significant increase in the proportion of participants in both the Heuristics-Between and Control groups who answered it correctly. The proportion of correct answers did not change between the pre- and post-assessments within the Heuristics-First group. It is possible that the amount of time that elapsed between the heuristics lesson and post-assessment played a role in this. The Heuristics-First group had the longest gap between the heuristics lesson and the post-assessment at around 30 min after the heuristics lesson; Ebbinghaus’ forgetting curve indicates a person will forget approximately 58% of information in that time span if it is not directly reinforced [53], so it is possible that other group participants were able to recall more of the information from the lesson while taking the post-assessment. However, the effect of the forgetting curve may have been influenced by the design problem acting as a recall cue for the DfAM heuristics, causing reinforcement of the learning.

Based on the results in Sec. 4.3, it was found that the quality of the designs did not differ between the experimental groups at the design phase or redesign phase of the study. This result does not support Hypothesis 1A, which anticipated that there would be a significant difference between the experimental groups at both phases of the study. None of the experimental groups experienced a significant change in quality from the design phase to the redesign phase. This result was expected for the Heuristics-First and Control groups, corroborating Hypotheses 1B and 1D, but it contradicts Hypothesis 1C, as it was expected that the Heuristics-Between group would experience an increase in the quality of the designs. One possible reason for these results is that the participants were overly familiar with the design problem, causing a lack of significant diversity in responses and little room for an increase in quality in the redesign session. Two other potential causes, which are discussed further in Sec. 6, are the high variance in scores and the virtual format of the study, which may have led to a decrease in effort or amount of attention paid to the heuristics lesson by the participants.

Deeper analysis of the quality subcategories resulted in the creation of a new merged category called Printable Quality, created from combining the Print Strength and Support Material subcategories, due to their immediate relevancy to 3D printing. Analysis of this category revealed some implications on design quality. Although the overall quality of the designs did not increase, the Heuristics-Between group did have a significant increase in Printable Quality, while the other two groups did not. This lends some support to Hypotheses 1C and 2; the latter predicted that the Heuristics-Between group would make more design changes than the other groups. This ended up being partially true, as the increase in Printable Quality indicated that the students from that group made changes to their designs to improve its ability to be 3D printed. This was additionally supported by the fact that 24 out of 29 participants from the Heuristics-Between group made changes to their design in the redesign phase, compared with 12 out of 20 participants in the Heuristics-First group and 12 out of 17 participants in the Control group. Hypothesis 1C predicted that the overall quality of the Heuristics-Between group’s designs would increase. While this did not occur, the Printable Quality of the group’s designs did increase. The heuristics lesson was focused on designing for AM, not on general design practices, so it does make sense that specifically the AM subcategories of the design would improve, while the non-AM-related categories did not experience a significant change. More specifically, the heuristics lesson focused heavily on DfAM restrictions, with only minor information on opportunities afforded by DfAM, possibly causing the participants to focus more strongly on the printability of their designs. Further analysis revealed that this increase in Printable Quality did not correspond to a change in the Functionality of the designs, showing that students did not sacrifice the functionality of their designs for easier 3D printing.

Similarly, the novelty scores did not differ significantly between the experimental groups, contradicting Hypothesis 3A. This may be explained by similar reasons to the ones stated previously: the soap dish is a simple product with a few well-established solutions; so many people already have an idea in their head of what a soap dish looks like. Therefore, participants may have been drawing from a somewhat finite pool of solutions, which limited the novelty of the designs that were produced. For example, one of the novelty categories was “Soap-Holding Method.” The results showed that there were only two main solutions that participants came up with to hold a bar of soap—(1) surrounding it with walls on some or all sides or (2) placing the bar of soap on a concave surface. This contributed to the lack of diversity in novelty scores between groups. Prior work has shown that more complex DfAM tasks result in designs with higher novelty compared with DfAM tasks with a comparatively simple design problem [54].

Upon closer examination of Hypothesis 3A, it was predicted that the Heuristics-First group would have the lowest novelty scores due to the priming effect of the heuristics lesson. Although the novelty scores between the groups were not found to be significantly different, the variety scores in Fig. 10 showed that the priming effect may have manifested itself in the Heuristic-First group’s designs. Of the 35 unique solutions to the novelty categories that were identified across all 66 sketches in the design phase, 27 of them were present in the Heuristics-Between and Control groups, while only 22 unique solutions were present in the designs of the Heuristics-First group. This finding, that there was less variation in designs within the Heuristics-First group, supports the notion that receiving the DfAM heuristics before the design session primed the participants toward choosing more similar designs.

As predicted in Hypothesis 3C, there was a significant decrease in novelty in the Heuristics-Between group. However, Hypotheses 3B and 3D were contradicted by the results; it predicted that the novelty of the Heuristics-First and Control groups would increase between the design and redesign phases due to the incubation effect [47], while the results indicated no significant change in novelty. As mentioned above, 24 out of 29 participants in the Heuristics-Between group created a new design in the redesign phase, instead of reusing their initial idea, which was more prevalent in the Heuristics-First and Control groups. Thus, the reason for the decreased novelty scores may be because the Heuristics-Between group spent the redesign phase coming up with new ideas in accordance with their newfound AM knowledge, thereby sacrificing novelty, instead of keeping the same idea. It is possible that the break did not function as an incubation period for the Heuristics-Between group because participants were focused upon the new information they had just received rather than the design problem.

Two of the observed Cura settings, filament usage and print time, seemed that they should be closely related, because a design using more filament will naturally take a longer time to print, in general. Examination of the mean values from each experimental group indicates that these variables followed a similar pattern when comparing the groups. However, this pattern was not statistically significant. Again, perhaps the design problem was too constrained: participants were given the dimensions of a bar of soap that needed to be held by their design, which likely constrained all of the designs to roughly the same envelope. As a result, the designs used similar amounts of filament and time to print, regardless of whether the participants had been exposed to the heuristics or not.

The Heuristics-First group used significantly more support material (as a percentage of total printing time) than the Heuristics-Between group. This result was unexpected, especially as the Control group’s use of support material was not significantly different from the other two groups. The Control group, having not been exposed to any DfAM heuristics before designing their part, was expected to use more support material than the other two groups. Support material usage depends on overhang features of design as well as how the part is oriented on the print bed. Although the control group completed their soap dish designs in class before receiving the heuristics lesson, it is possible they applied some of the heuristics from the lecture when determining how to orient their parts on the virtual print bed.

Despite the potential contributions to the field, this study had its limitations that must be considered. Many of these limitations are mentioned in Sec. 5 as possible explanations for unexpected data trends. Some were related to the experimental design, while some were unavoidable. The most obvious uncontrollable limitation was the virtual format of the study, due to health and safety measures taken in the class during which the study was conducted. It is possible that the virtual format of the study contributed to its unexpected results. Students have been found to be more likely to multitask during online classes than during in-person courses [55]. It is possible that participants were distracted or multitasking during the study, meaning they may not have given their full attention to the design activities or the heuristics lesson. Future work could involve running an identical study in an in-person format and comparing the results to those obtained from the virtual-format study. In future studies, an effort will be made to increase study recruitment in order to have larger sample sizes, as well. Group sizes in this study were limited by course enrollment.

Among the quality scores in particular, the variance in scores was quite high, which may have contributed to the reason the effects of time and experimental groups were nonsignificant. Based on observation of the designs, this seems to have occurred because several participants set out to design an easily printable soap dish without considering whether it would meet the two required functions. Because functionality was worth 50% of the quality score, and many participants received a neutral or negative functionality score, this resulted in high variance between the quality scores, as shown by the error bars in Fig. 4. A source of potential uncertainty is the subjectivity involved in the evaluation of design quality, particularly the assessment of “Print Strength” and the ad-hoc determination of the thresholds for “Interfacing Items.” Another contributing factor may have been the significantly larger sample size of the Heuristics-Between group, which may have impacted the power of the effects. In future studies, an effort will be made to keep sample sizes as equal as possible.

If this study were to be repeated, the Control group would be given the heuristics lesson after completing the post-assessment. Responses to the post-assessment may have suffered from response biases such as demand characteristics, identified by Orne [56] as the desire for participants to be a “good subject”. Study participants tend to respond in ways that will confirm the hypothesis [57]. Therefore, it is not clear how much of the participants’ self-efficacy score increases were due to a genuine increase in knowledge, and how much was due to an attempt to meet the perceived expectations of the researchers. Giving the post-assessment to the Control group before the heuristics lesson would have permitted this effect to be quantified, so that it could be determined how this affected the other experimental groups. Additionally, post-assessment performance related to heuristics may have been impacted by the primacy or recency effect, meaning that participants were better able to recall heuristics that were presented towards the beginning or end of the lecture [58]. This may have also impacted the way participants applied heuristics to their sketches, as previous work on this topic has found that presenting opportunistic DfAM knowledge after restrictive DfAM knowledge, as was done in this study, results in more creative designs compared to groups that received DfAM knowledge in the inverse order [59].

As 3D printing technologies develop, it becomes more difficult to find subjects who are true novices to the technology. Some of the participants reported owning a 3D printer for personal use, or working in the institute’s makerspace with access to dozens of 3D printers. Although this experience was distributed evenly between the experimental groups, it may have impacted the study. Students who had never used a 3D printer before were likely to learn more from the heuristics lesson than those who maintained 3D printers in the makerspace.

The break between the design and redesign sessions was a source of uncertainty when analyzing results. Yang et al. [46] found that a break that is too long or too short does not provide a creative advantage, with the optimal break length being three minutes. In this study, the break was 5 min long. This was further complicated by the fact that for the Heuristics-Between group, 40 min elapsed between the design and redesign sessions, compared with 5 min for the other two groups. As a result, the Heuristics-Between group almost certainly experienced a difference in unconscious creativity compared with the other two groups. Ideally, the time between the design and redesign sessions could have been adjusted to allow for a 40 min break between the sessions for all three experimental groups, but this was unfortunately not possible due to time constraints in the virtual classroom.

The work presented in this paper has contributed to the literature on design heuristics for additive manufacturing in the ways detailed below.

This work provides a quantitative analysis of the effects of exposing novices to DfAM heuristics, especially on the effects of quality, which makes it a valuable contribution to the study of applied heuristics. From the analysis of the printable quality scores, it can be seen that exposing novice designers to DfAM heuristics can lead to the formulation of designs that are better-suited for 3D printing, even if the designers have no prior 3D printing experience. Additionally, results from assessments indicate that showing DfAM heuristics to novices improves their perception of their ability to perform AM-related tasks.

Because the study was conducted in a classroom setting, the results can be applied in an education context. Analysis of novelty scores shows that to encourage novelty, it is important to provide time for designers to revisit and iterate upon their designs without being required to process new information. This is especially critical in an engineering education context when teaching undergraduate students to fully utilize the iterative power of the design process. As the additive manufacturing field develops rapidly, it is important for designers to understand how to use new technologies.

The main research question identified in this paper was

This question was addressed by the results and discussion of the DfAM heuristic study. Specifically, it was found that participants who received the heuristics lecture in between the two design sessions produced designs that were better-suited for 3D printing in the second (redesign) phase of the experiment compared with the first (design) phase. However, the overall quality of the designs produced did not vary significantly between experimental groups or between phases of the design activities. Participants who received the heuristics lecture either before or after both design sessions produced similarly novel designs in the design and redesign phases, but receiving the heuristics lecture between design sessions appears to stifle novelty. In addition to its impact on novelty, the heuristics lesson caused a priming effect in the first group of participants, which can be seen by its relatively low variety score in the initial design phase of the study. These findings are important because they give some insight into the way novices use DfAM heuristics as a design tool and indicate that although heuristics are a valuable resource for training and education, care must be taken to promote design iteration and avoid priming and design fixation.

I would like to thank the students of ME2110 for participating in this study, as well as their instructors, especially Dr. Chris Saldana, for allowing me to conduct this study as part of class.

• This work was supported by the U.S. Department of Energy (DOE) award DE-EE0008303.

There are no conflicts of interest.

1. Restrictive DFAM heuristics

   • Overhang: If there is an overhang on the part, ensure that the angle is smaller than 45 deg. If the angle of the overhang is larger than 45 deg, then support structure is required.

   • Accessible support: If your part requires support structures that must be removed, make sure they are not trapped inside an inaccessible volume. If support structures are trapped, modify the part geometry to ensure it can be removed.

   • Part size: If the part is larger than the build area in one direction, either reorient it or split the part into two.

   • Corners: If the corners on the bottom of the part are warping, design the part to have rounded (filleted) corners instead of sharp corners.

   • Surfaces: If the angle of a surface is not 0 deg or 90 deg, the surface will have a staircase effect. This can be fixed with surface machining. Reducing layer height will also improve the surface roughness of the part. Redesign the part with faces that are parallel and perpendicular to the print bed.

   • Feature Size: Rule of thumb: features should be > 1.0 mm in thickness. Minimum feature thickness depends on printer specs: 0.6 mm (X–Y) and 0.1 mm (Z).

   • Infill: The inside of the part should probably not be 100% infill (depending on the function of the part) in order to reduce printing time and material cost.

   • Part orientation

     • Support material: To avoid needing support material, orient your prints in a way that minimizes overhangs

     • Surface finish: Optimize the orientation of the model to avoid the appearance of stepping in the z-direction

     • Load: If the 3D printed part will bear a load, consider the application and direction of the load when deciding how to orient the part.

     • Surface area: Design parts so that the amount of surface area in contact with the print bed is maximized. Ideally, this will be a large, flat face.

2. Opportunistic DFAM heuristics

   • Conveying information: If you want your part to convey information to the user, consider conveying it with: color, geometry, haptics, light.

   • Consolidating parts: If you are considering 3D printing parts of an assembly, try to consolidate the parts.